Intro to ClojureScript - Part 3 - Using Shoreleave-pubsub

In this post we will move away from the function based event handling we put in place in my previous post to use Shoreleave’s pubsub library. Using pubsub will allow use to easily add and remove functionality over the lifetime of our app. Once we’ve replaced the current tightly coupled event handling we will add console logging and the ability to view our search history.

The pubsub library works like the name implies, topics are published and can be subscribed to. Topics can be strings, keywords, functions, atoms, web workers or local storage. In this post we will use keywords to indicate when a search is to be performed, when the results come back, and when the view history button is clicked. An atom will be used to track the current search and keep a list of previous searches.

Since ClojureScript is a bit of a mouthful I will use the abbreviation cljs for the remainder of this post. Remember I am learning cljs as I create these posts so any and all guidance is appreciated.

The Goal

By the end of this post we will have converted the event handling code over to use shorelave-pubsub and have added console logging and the ability to view our search history for the current session.

The Setup

If you don't already have cljs installed on your system go read Intro to ClojureScript - Getting Started to get your environment ready for cljs development. You will also need the code from my previous post. If you don't already have it you can grab it from my cljs-intro project on the part-2 branch. Once your environment is setup we can start making the necessary changes to add pubsub support.

Project.clj Changes

We only have one change to make in the project.clj file and that is to add the reference to the shoreleave-pubsub library:

[shoreleave/shoreleave-pubsub "0.2.2"]

Once the change is in place save the file and run lein deps.

Index.html Changes

We need to add the 'View Search History' button to our web page so users can view their previous searches. Add the new button next to the ‘Get Stats!’ button. The updated index.html should look like this:

After updating the HTML we are ready to get into the cljs changes. The first step is to create the new file that will contain the pubsub code.

The PubSub Code

A little background

Before we get into the pubsub code lets take a minute to go over shoreleave-pubsub. The shoreleave-pubsub library uses two protocols IMessageBrokerBus and IPublishable to provide the necessary functionality. The IMessageBrokerBus manages the bus, giving us the ability to publish, subscribe and unsubscribe to a given bus. The IPublishable protocol provides us with functions that define things that can be published. We will use the topicify and publishize functions to convert the keywords and atom to topics. For a more detailed look into what these two protocols provide read the Shoreleave pubsub docs.

src/cljs/pubsub.cljs

As you might expect the pubsub.cljs file is where we’ll put our pubsub related code will live. The file is broken down into three areas: config, helper functions and subscriptions.

config section

The first thing we need to do is require two namespaces shoreleave.pubsub.simple and shoreleave.pubsub.protocols. shoreleave.pubsubs.simple provides us with a synchronous bus which is fine for this example. The shoreleave.pubsubs.protocols library provides the functions we need to turn our keywords into topics and make our search-state atom ‘publishable’.

First we create the bus and then we ‘topicify’ the three keywords, :search, :results, and :history. Once the keywords have been 'topicify-ed' we can use them to publish messages. The search-topic will be used anytime a user clicks the ‘Get Stats!’ button. The results-topic will be used when that stats are returned. As you might expect the history-topic will be used to indicate that the user has clicked the ‘View Search History’ button.

Turning the search-state atom into something that can be used as a topic is slightly different. We need to call the publishize function passing in the atom and the bus it will be published on. Calling the publishize adds a watch function that publishes all changes to the atom’s subscribers. The published data contains a copy of the new and the old state of the atom. Subscribers can access the data by using the either the :new or :old keyword.

helper functions

The helper functions are there to keep the ‘guts’ of the pubsub code out of the search.cljs file. The publish-* functions are there so code in the search.cljs can publish messages to all subscribers. The subscribe-to function wraps the pubsub subscription process so subscriptions can be created outside of the pubsub.cljs.

subscriptions

The last two lines of the file create subscriptions to search-topic and search-state. These subscriptions, subscribe the anonymous functions to each topic. Now, anytime there is a message published on either one of these topics the published data will be written out to console.log.

src/cljs/search.cljs

The search.cljs file underwent quite a few changes during the conversion. Anything to do with event handling changed as did some of the functions called from within the event handlers. In addition to coding changes we had to add a new require statement for the new namespace cljs-intro.pubsub.

Subscriptions and Event handling

Since we are using pubsub to handle events and state changes we need subscribe to the topics we are interested in.

The subscription calls are using the helper function subscribe-to which takes the topic you want to subscribe to and the function that will be called when a new message is published.

Now when a handler is called instead of calling the function to handle the event, a message is publish for a specific topic. For instance, when a user clicks the ‘Get Stats!’ the handler publishes a message on the topic search-topic passing the value that was entered by the user to all subscribers. Here’s the difference between the two versions:

When we subscribed to the search-topic topic we established that the player-lookup function would be called. There was only two changes made, the first is to publish the results of the remote call when it returns instead of calling display-results directly. The published data is converted to a Clojure data structure prior to publishing freeing the subscribers of any unnecessary steps prior to using the data.

The display-results function is subscribed to the results-topic. The new display-results receives the converted data so we no longer have to convert it before we display it. I’ve also created a new function, update-results-div, to handle the updating of the search results.

The functionality we just walked through is the same as it was before we switched to using pubsub. You may be wondering whats the point? The conversion to pubsub allows us to open up the application so that adding new functionality can be done with little to no changes in the search.cljs code. As an example, we added console logging for the search string by adding a subscription to the search-topic and giving it a function that calls js/console.log. No changes were required to the core search code. The image above shows the search logging and the logging that occurs anytime the search-state atom changes. Thank you pubsub.

Viewing Search History

Another piece of functionality that was added is the ability to see the previous searches you’ve run.

When does search-state change? Every time a new search is submitted the current value of :lastname is conj’d into the :previous-searches sequence provided that :lastname is not nil or empty. When that happens a new message is published to notify all subscribers of the change. In our app we have subscriptions to search-state for managing the visibility of the ‘View Search History’ button and for logging.

The Code

When search-state is changed the search-state-change function is called. This function is where the visibility of the ‘View Search History’ button is determined. If the :previous-searches sequence contains at least one string then the domina set-styles! function is called to make the button visible.

The history button’s click event handler publishes a message on the history-topic. Our subscription for this topic calls cljs-intro.views/view-history function which takes the :previous-searches sequence as its only parameter. From that sequence an unordered HTML list is created and displayed using the new function update-results-div. Here’s what it looks like after a few searches and a click of the ‘View Search History’ button.



Summary

We now have an app that uses shoreleave-pubsub for handling events instead of the traditional, tightly coupled event handler functions. Using the library has allowed us to add new search related functionality without making any changes to the core part of our application. Now we have a more flexible application.

Whats Next?

In Intro to ClojureScript - Part 4 - Using Shoreleave’s Remoting we will convert the current remote callto the server to use shoreleave-remoting. Part 4 may not be up until two weeks due to an overbooked schedule. I will be speaking at Richmond Code Camp giving an ‘Intro to ClojureCLR’ talk and I need to get started on my talk. As soon as thats done Part 4 will be up.

Resources

shoreleave-pubsub docs. I spent a lot of time reading the docs for shoreleave-pubsub.  

Hockey Databank Database -- I created the database using data that originates from a Yahoo! group called hockey databank. After each season the group produces updated CSV files with the stats of the previous NHL season in addition to previous years data.

ClojureScript Experience Report - Resources Bits of information about Jason’s ClojureScript based application and his experience with ClojureScript.

ClojureScript: Up and Running - An early release book that discusses ClojureScript. I’ve read the released chapters and found it to be a good resource. Chapters 2 and 3 go over ClojureScript’s compilation, project structure, and other informative tidbits for both ClojureScript and Clojure. I’ve recommended the book to a colleague who is learning Clojure because of the way the authors describe data structures, immutability and sequences.

cljs-intro - My github repo that houses the code for this blog series. Each part of the series has its own branch. This post’s code is on the Part 3 branch.




Intro to ClojureScript - Part 2 - Getting the Stats

In this post we will flesh out the backend of the app we started in Intro to ClojureScript - Getting Started. We are going to create a web service to lookup player stats by last name. Once we have the lookup service in place we will update search.cljs to fetch the results of our query using the Google closure library’s goog.net.XhrIo. When the data is returned we will display it to the user using the hiccups library.

Since ClojureScript is a bit of a mouthful I will use the abbreviation cljs for the remainder of this post. Remember I am learning cljs as I create these posts so any and all guidance is appreciated.

The Goal

By the end of this post we will have a web based app that will lookup player stats by last name, displaying the results of our search.

The Setup

If you don’t already have cljs setup on your machine read my Getting Started post. It will walk you through the necessary steps to set up your cljs environment. I am also assuming you have the code from the first post. If not you can grab it from my cljs-intro github repo.

Grabbing the DB

We will be using a SQLite version of my hockey stats database. You can download it from my hockeydb github project. Once you have the database, create a db directory in the project’s home directory and copy the downloaded database into it.

Project.clj Changes

Since we will be interacting with a database, manipulating the page’s DOM, and creating a web ‘service’ we need to add a the following dependencies to our project:

[org.clojure/java.jdbc “0.2.3”]
[sqlitejdbc “0.5.6”]                  
[domina “1.0.0”]
[noir “1.3.0-beta0”]
[hiccups “0.1.1”]

The first two libraries are for our database interaction. The domina library allows us to manipulate the DOM from our cljs code. We will use noir to create our lookup web service. The hiccups library will be used to add and remove HTML from the web page.

The last setup step is to update the resources/public/index.html file. Most of the changes in it are for twitter bootstrap support. Adding bootstrap will make the search results a little easer to read.

The only change needed for the application is the addition of results div. As you might of guessed this is where the div that will contain the search results.

Creating the DB Access Code

All of our database related code will live in src/clj/cljs_intro/db.clj file. The code does a few simple queries to gather the player related information. I’m not going to go through this code in detail since it is ‘normal’ Clojure and not cljs. If you'd like to see the code here's the link to at github: db.clj.

In the db.clj file I’ve made the executive decision to return the stats for the first matching player. Why? I don’t wan’t to spend a bunch of time on non-cljs code so I made it simple on myself. What does that mean? It means if you search for Gretzky you’ll get Brett Gretzky instead of Wayne. I bet you didn't know there was a Brett Gretzky did ya.

Creating ‘Web Server’

Now that we have the database its time to create our simple web server. Noir provides functions that allow us to respond to HTTP requests, mapping routes to code and return JSON data. Our server will listen on the port 8888 for following two URLs:

http://localhost:8888/index.html
http://localhost:8888/player/:lastname.

The last URL has a parameter (:lastname) which will be the player’s last name entered by the user. Any other request will throw a Noir error.

Now that our server is complete we need to update the project.clj file so we can start it by running `lein run`. Add the following to the project.clj file:

:main cljs-intro.core

Lets test the server out. At the command prompt enter:

lein run

When the server starts up open your favorite browser and go to http://localhost:8888/player/ricci . If everything goes well you should see about a page full of JSON data.

Retrieving and Displaying the Data

If you go to http://localhost:8888/index.html and enter ricci in the text box and click the ‘Get Stats!’ button you’ll see an alert box with ricci in it. That was fine for part 1 but now we are ready to get the stats. In order to do that we need to update the ‘Get Stats!’ click event handler. Instead of a js/alert call we will call our player-lookup function.

The Updated Click Event Handler

The player-lookup function wraps a call to the Google Closure library’s goog.net.XhrIo.send method.

The goog.net.XhrIo.send function has two parameters, the URL to call and a callback function to process the returned data. We create the URL parameter by concatenating the base URL (/player) with the user’s input, in our example its ricci. When the call returns the data is passed to the display-results function so we can, well display the results.

display-results

The display-results converts the returned JSON data into a Clojure data structure and manages the UI updates.

The call to js->clj coverts the JSON to a Clojure data structure. A key thing to note here are the parameters :keywordize-keys true. Before I used those two parameters I was having a hell of a time getting to the data. After using the parameters I could easily access the data using keywords. I’m assuming that most of the time you use js->clj you’ll want to add :keywordize-keys true.

The next line uses the domina.xpath/xpath function to get a reference to the results div DOM object. We will use the reference to display the search results.

Before displaying the results of the current query we remove any previous query results by using the domina function called destroy-children!. The parameter for the destroy-children! function is a reference to a DOM object with the children to be removed. Now that the display area is empty we can display the results of the new search. On the last line a call is made to cljs-intro.views/show-stats which is where the HTML generation process starts. The results of that call are appended to the results div by calling domina’s append! function.

The HTML

Creating the HTML is done using the hiccups library which is a port of the Clojure hiccup library. Since I've used hiccup and noir for non-cljs web-based apps using hiccups made the HTML generation easy. The only difference I saw between hiccups and hiccup was that I had to use defhtml when creating a function that generated HTML.

If you wish to view the code that generates the email you can see the views.cljs. Now, when we run the search for Ricci we see the following page displayed:



Summary

The search app makes a remote call using the Google Closure library. When the data returned we were able to convert the JSON to a Clojure data structure. Once the conversion was complete we used the domina library to remove and add HTML with help from the hiccups library. Our application actually returns the stats for the player (as long as you only want the first person with the last name in question).

Whats Next?

In Intro to ClojureScript - Part 3 - Using Shoreleave - we will use the Shoreleave libraries to decouple the front end from the back end by using event based messaging. We will also change the way we handle the remote calls to use the shoreleave-remote library.

Resources

twitterbuzz and this stack overflow post implementing an ajax call in clojurescript helped me with the remote call.

ClojureScriptOne.com - used for DOM manipulation guidance.

Hockey Databank Database -- I created the database using data that originates from a Yahoo! group called hockey databank. After each season the group produces updated CSV files with the stats of the previous NHL season in addition to previous years data.

ClojureScript Experience Report - Resources Bits of information about Jason’s ClojureScript based application and his experience with ClojureScript.

ClojureScript: Up and Running - An early release book that discusses ClojureScript. I’ve read the released chapters and found it to be a good resource. Chapters 2 and 3 go over ClojureScript’s compilation, project structure, and other informative tidbits for both ClojureScript and Clojure. I’ve recommended the book to a colleague who is learning Clojure because of the way the authors describe data structures, immutability and sequences.

cljs-intro - My github repo that stores the code for this blog series. The code for Part 1 can be found on the branch Part 1. This post’s code is on the Part 2 branch.

Intro to ClojureScript - Getting Started

I’ve been curious about ClojureScript for awhile now but for whatever reason I hadn’t taken the plunge. A few weeks back I attended a at talk at a TriClojure meetup titled ‘A ClojureScript Experience Report’. Jason broke down his experience creating an app using ClojureScript ins such a way that it motivated me to start learning ClojureScript.

This is the first in a series of posts that I will write that chronicles my journey down the ClojureScript path. I will be building out a basic app to lookup hockey stats by the last name of a player. Keep in mind I’m learning ClojureScript as I write these posts. If you have experience with ClojureScript and you see me making errors or not doing things in a idiomatic way please feel free to point out the errors of my ways.

Since ClojureScript is a bit of a mouthful I will use the abbreviation cljs for the remainder of this post.

The Goal

By the end of this post you will have an environment that is ready to create cljs apps. You will also have the beginnings of our hockey stats lookup app.

The Setup

Before you can start writing you app you’ll need to get your environment setup. The first step is to get Leiningen installed. Follow the directions on the page and you’ll be ready to create your first project.

Creating a cljs project is done the same way you create a Clojure project:

lein new cljs-intro

This will create Clojure project with the following layout:

README
project.clj
src
    cljs_intro
         core.clj
test
   cljs_intro
         test
               core.clj

In order to keep our Clojure and cljs code seperate we need to change the directory layout of our project a little bit. We need to add two directories, move the existing Clojure code and create the directory for housing our HTML and cljs compiler output. If you are in the project’s home directory run the following commands:

1. mkdir src/clj src/cljs
2. mv src/cljs_intro src/clj
3. mkdir -p resources/public

Now that we’ve created the directories and moved the code we need to update the project.clj file to reflect these changes and add cljs specific settings.

The project.clj file generated by the lein command looks like this:

First we need to add the plugin lein-cljsbuild to the project. lein-cljsbuild is the plugin that makes it possible to compile cljs apps. Adding lein-cljsbuild to our is as simple as adding the :plugin line to the project file. Here’s what the updated project.clj file looks like after adding the line.

Since we moved the source code location we need to update the project.clj file to reflect those changes. By adding the following line the file we inform lein where the Clojure code lives.

:source-path "src/clj"

Now its time to set up the cljs related settings. Here’s what we need to add to the project file:

Here we are we are telling the cljs compiler to output the results of the compilation into the file resources/public/hockey.js. Since we have an optimization setting of :whitespace all compiled javascript will be placed into a single file. There are four levels of optimization levels available: :none, :whitespace, :simple and :advanced. We will stick with the :whitespace option. The :whitespace setting removes comments and any unnecessary whitespace in the source code but leaves the remaining code alone. The book ClojureScript: Up and Running has a nice discussion about optimization. The :pretty-print option gives us output that is a little easier to read. Here’s what the updated project.clj file looks like:

To ensure we have everything correct in the project.clj file go ahead and start the Clojure repl by running this command:

lein repl

If everything is OK you should see something like this:

REPL started; server listening on localhost port 42236
user=>

Time to Write Code

Now that we have our project setup for cljs its time to start writing some code. For this post we will create the search form which will have a text field and a button. The ‘back end’ of the form will be written using cljs and will be responsible for responding to the search button click. When the button is clicked the cljs code will grab the value of the text field and displaying it in an alert box.

resources/public/index.html

Before we write the cljs code lets create the HTML page. Its a simple form that has a text box and a button. Here’s what the HTML looks like:

The only real line of note here is the script tag. It references the file that we will generate when we compile our cljs code.

src/cljs/search.cljs

The src/cljs/search.cljs file is where we will put the code to handle the user’s input. Once you have created the file open it in your favorite editor and add the following lines:

The above code sets the namespace for the code and brings in the domina and clojure.browser.event libraries. We need the domina library so we can interact with the DOM. The clojure.browser.event library is used to add the event handler for the button’s click event.

Before we write the search button's click event handler lets get a reference to the search button itself.

(def search-button (d/by-id "search-btn"))

We get a reference to the button by making use of domina's by-id function and store the reference to the DOM object in search-button. Now its time to create the event handler.

The clojure.browser.event/listen function creates the relationship between the search button’s click event and our handler function. We create a function to popup an alert box containing the contents of the text field.

Compile Time

To compile the cljs code we have two options: lein cljsbuild once or lein cljsbuild auto. The once option compiles our code like a ‘normal’ compiler. It runs, compiles the source and puts the output to the file that we indicated in the :output-to setting. If we compile with auto every time a cljs file is saved the compiler will run. For now we’ll stick with the once option. To compile our code go to anywhere in the project’s directory structure and enter:

lein cljsbuild once

Now open up the the HTML file we created in the browser and you should see a beautifully designed form that looks something like this:

Enter the string ‘ClojureScript!’ in the text field and click the ‘Get Stats!’ button. You should see something like this:

We now have the beginnings of our stats lookup app. The user interaction is in place but we don’t really have a lot of the ‘under the covers’ code in place. In my next post we will add code to the hockey database and run an actual query!

Summary

In this post we went over how to get your environment set up for ClojureScript. We also went over the necessary changes to the project directory structure and project.clj file. We also created a simple cljs file to handle the user’s input in our search form.

Whats Next?

In Intro to ClojureScript - Part 2 - Getting the Stats we will create the necessary back end to retrieve data from the database and display it on the page.

In Intro to ClojureScript - Part 3 - Incorporating ClojureScriptOne - We will add support for messaging. We will add a new feature with messaging.

Resources

ClojureScriptOne.com - A sample project to help people get familiar with ClojureScript.

ClojureScript Experience Report - Resources - Bits of information about Jason’s ClojureScript based application and ClojureScript.

ClojureScript: Up and Running - An early release book that discusses ClojureScript. I’ve read the released chapters and found it to be a good resource. Chapters 2 and 3 go over ClojureScript’s compilation, project structure, and other informative tidbits for both ClojureScript and Clojure. I’ve recommended the book to a colleague who is learning Clojure because of the way the authors describe data structures, immutability and sequences.

cljs-intro - My github repo that stores the code for this blog series.

Creating a TFS Work Item from ClojureCLR

At my day job we use Team Foundation Server 2008 (TFS) for our automated builds, iteration management, and source control. TFS may not be the most ideal way to manage these processes but in our MS environment it has helped us communicate with our non-technical team members and customers.  So we’ve been looking into ways to add bugs automatically when automated tests fail (see Creating A TFS WorkItem From IronRuby) or errors are reported from our applications via a message queue.

This morning I had a little time to do some ‘research’ on how to programmatically create a new bug work item.  My goal was to write code that would create a new work item bug with an image attached to it.  Before I get into the code let me take a second to introduce you to a TFS Bug Work Item.

A TFS Bug Work Item

In TFS Work Items are a way to track work that needs to be done. There are a total of five different type of Work Items available but we typically only use three: Task, Scenario, and Bug. Each Work Item type UI has different fields. Here is what a Bug Work Item looks like in our environment.

In this example we will create a new bug and enter text into the highlighted fields plus attach an image file. In order to create the bug I need to do a little setup.

The Setup

There are three assemblies needed to create a TFS Bug Work Item. The are: Microsoft.TeamFoundation.dll, Microsoft.TeamFoundation.Client.dll, and Microsoft.TeamFoundation.WorkItemTracking.Client. All three of these DLLs can be found in the C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies directory. I copied the three Dlls into the project’s libs directory.

The Code

The first step is to load the TFS assemblies from libs directory using the assembly-load-from function. I follow that call up with the import function pulling in the specific classes I need.  Once the assemblies and classes have been loaded I can start working on creating a bug work item. First, I created the -main function. The first few lines of –main are responsible for creating the necessary objects for WorkItem creation.

The first line gets the server object by making a call to the static method TeamFoundationServerFactory.GetServer passing in the name of the server I am working with.  The returned server object is used to create a WorkItemStore object.  A WorkItemStore encapsulates the data store that contains all work items on a particular server.  The next line gets the TFS project that I want to work with by using the filter function.  The last setup line uses the project object to retrieve the WorkItemType object that represents a bug.  Now I 'm are ready to create a new bug WorkItem.

Creating the Bug WorkItem

Since I will be creating bugs that have attachments and bugs that do not I created a function that creates and returns a WorkItem object. As I stated previously, the work items created this way will have the bare minimum values: title, area, iteration and description. The code is straight forward, just create the WorkItem object by passing in the WorkItemType object. Next I set the properties of for the four parameters. When everything has been set I return the WorkItem object.

Adding the Attachment

Now that I have my WorkItem object its time to add my image file as an attachment. I wrote another small and simple function to create the Attachment object. It takes three parameters, the WorkItem, the path to the file to attach and a description of the file.  First I use the path and desc parameters to create the Attachment object.  After the Attachment is created I can added it to the WorkItem object’s Attachments collection by calling the Add method passing the Attachment object as its only parameter. 

At this point I have a WorkItem object with an Attachment in memory.  At this moment the WorkItem is not visible through the TFS toolset.  I still have one last line of code to write to ensure that the WorkItem is saved. Surprisingly enough all I need to do is call the WorkItem’s save method.

(.Save work-item)

Now, If I look at the Pending Bugs report in TFS I will see my newly created bug in the list.

Summary

This post is another illustration of how easy it is to use .NET Assemblies from ClojureCLR.  In this case I walked you through the simple process of creating a Bug WorkItem in TFS.  Having the ability to programmatically create and report bugs will help us close the feedback loop in our applications. 

In The Future

In my ‘free time’ I will be checking into TFS’s build and source control APIs to see what we may be able to do improve our processes.  If and when I find anything of interest I will be sure to do a post or two about my findings.

Resources

TFS API: http://msdn.microsoft.com/en-us/library/bb130146(v=vs.90).aspx

My Source (this blog’s code is the 5-create_tfs_work_item):https://github.com/rippinrobr/clojure-clr-intro/zipball/master

My First Attempt at Porting Libraries to ClojureCLR

A while back David Miller wrote a couple of blog posts (Porting libs to ClojureCLR: an example and Porting effort for Clojure contrib libs) about porting Clojure contrib libs to ClojureCLR.  In the two posts he made it look like porting most of the libs wouldn’t be too terribly hard.  After a long period of telling myself, “tomorrow I will pick a lib to port and get started”, I actually did do just that a few days ago. I am about to start a project that needs to process command line arguments and it just so happens that David flagged the tools.cli lib as a trivial port.  So I decided to start my porting career with tools.cli.

The Setup

I created a directory off of my usual code directory called clj-ports just to help me differentiate that code from everything else I’m working on. Next I cloned the existing github project (https://github.com/clojure/tools.cli).  After cloning the project I attempted to compile the project with clojure.compile. For tools.cli the compilation went off without a hitch. Now that I have a clojure.tools.cli.clj.DLL  I’m ready to start testing.

The Testing Process

Now that I have my DLL I’m ready to start up the REPL and run the tests. The first step to running the is to load the clojure.test and the clojure.tools.cli-test libraries. After loading the libraries I can run the tests.  When I ran the tests and very quickly I hit my first error.

The error was a System.MissingMethodException on a call to a split method. I scoured the cli.clj file for a call to split but couldn’t find one. After taking a closer look I realized it was in the TESTING code. I opened up cli-test.clj, changed split to Split. When I re-ran the tests I hit another error. This time the test failed because it expected a Char[] and was passed a String.

The problem is in the testing code and is one of the parameters to the Split method call.  Since it was looking for a Char[] and I’m passing in a one character string I thought I’d try to use (first “-“) to get around the issue.  So I updated the test code and the change comment and re-ran the tests.  The error message I received this time was in the cli.clj file and not the test file.  Now that I have resolved the issues in the test file I can add my end of line comments following David’s instructions in his post Porting effort for Clojure contrib libs.

The new error is:

actual: System.MissingMethodException: Cannot find member startsWith matching args at .CallSite.Target (:0)

I opened up the cli.clj file and searched for startsWith calls. I found four places where the java startsWith method was used.  I changed all four to use the .NET equivalent StartsWith (and of course I commented the lines accordingly).  Remember, after you make a change to the source file you will need to re-compile (at least that is the way I did it) which means you’ll need to exit out of your REPL session first.  After recompiling and getting my REPL session back in order I re-ran the tests.  This time they all PASSED!

Now that I have all greens on my test it is time to create a new project on github called cljclr.tools.cli. After creating the project on github and setting up my machine I copied all files from the clojure.tools.cli project into the new cljclr.tools.cli project.  Next, I changed the namespaces in all code files to reflect the new project name and re-ran the tests.  All green! I checked the ported code into github and called the port complete. The ported project lives at cljclr.tools.cli.

Sample App

Since this was my first time porting a library I wanted to make sure it worked I used the ‘application’ above to check.  The first part of the main function is where the call to the cli function is which is where the available command line options are configured. The parameters are the args passed to the main function followed by a vectors that illustrate the allowed command-line options.  The returned vector has three members, a map with the parameters and their values, a vector with any additional parameters passed on the command line, and the string that contains the ‘banner’ which is used to show the available parameters.  After the call the cond function  is used to figure out what to do.  Each option has a simple println if one of the available parameters is passed in.  If no parameters are passed the usage banner is printed.  After compiling and running the app I was convinced that the port worked! 

Since That was Easy…

I thought I’d try my hand at another lib.  This time I chose to port tools.macro.  I went through the same configuration process, cloning, compiling and starting up the REPL so I can run the tests.  This time I hit NO errors when I ran the tests. That is the easiest type of port! 

Like a gambling addict I pressed my luck and chose one more library to port.  Since the algo.monads library was flagged as trivial I grabbed it.  When I ran the tests I was hoping for the same results as the tools.macro port but this time I hit an error.  The first error was another simple swap a java class for a .NET class: java.lang.IllegalArgumentException for System.ArgumentException and re-ran the tests.  The second error was in an extend-protocol call  using java.lang.String.  I simply swapped the java.lang.String for a System.String call.  When I ran the tests again all tests passed.  Another port complete!  The project can be found at cljclr.algo.monads on github.

Summary

Porting libraries to ClojureCLR wasn’t nearly as time consuming or complicated as I thought it would be.  In fact I have probably spent more time on writing this blog post than I did doing the ports.  Of course I picked libraries flagged as trivial ports which is probably why it didn’t take long to finish.  What porting these libs over did teach me was the process: Clone, Compile, Test, fix, retest.  For my next porting attempt, the data.csv library I will add a step to search the cloned files for easy fixes like startsWith to StartsWith and other mappings that David laid out in his posts.  Please join the porting party and port one today!  I have not tested the ports against 1.2, all of my ports were done with 1.3.  I will continue to test only with 1.3 unless others feel that we need to test against 1.2.  If you use any of the libraries I’ve ported and run into a bug please do not hesitate to let me know.

Resources

Blog Posts: Porting libs to ClojureCLR: an example and Porting effort for Clojure contrib libs

Ported Projects: cljclr.tools.cli and cljclr.algo.monads

Projects that work ‘As Is’: clojure.tools.macro

In order to help others see which libraries have been ported and/or tested for ClojureCLR I have created a page that lays out which libraries have been ported and the location of the github project. If you have ports you’d like listed on the page please let me know. The ClojureCLR Ports Status Page is here.

Intro to Clojure-clr: Creating a UI with Windows.Forms

In this post I will walk you through the creation of a Windows.Forms based user interface using ClojureCLR.

The Setup

To follow along with the blog you will need to have ClojureCLR installed and have access to a MySQL and SQL Server instances. If you haven’t installed it yet follow the directions in my Getting Started with Clojure-clr post. If you need to install one or both of the bases you can download and install them from here: SQL Server Express and MySQL.  Once you have downloaded and installing the databases the last step you’ll need to take is to grab the mysql_schema.sql file and run it in your MySQL database. If you are not familiar with MySQL you can run the file from the command line by running this:

mysql –user=username –password=password < mysql_schema.sql

This will create a database and three tables: Master, Schools and PlayerSchools. The three tables will be migrated into a new SQL Server database. These tables are from a database created by the group baseball-databank.org who have collected baseball statistics from the beginning of professional baseball to the present. I find using this data more exciting then the usual demo data.  Anyway, once you have the loaded the schema into your MySQL instance you are ready to start building the user interface.

UI.clj

Surprisingly enough the ui.clj file is where the user interface code lives. Here’s what it looks like when it is first started.

I won’t bore you with a line by line analysis of the source but I will show you highlights of the UI creation process, walking you through the steps below. Creating Windows.Forms apps follow the same process without regard to the language that they are being creating in. So there aren't any magic steps here. However, creating the UI in Clojure was a lot of fun.

Step 1. Including the necessary assembly and classes

The first step in creating the UI is to bring in the System.Windows.Forms assembly which I did by calling LoadWithParitalName. Next, I used :require to bring in the classes I needed create the UI with. Since I want to be able to run this app outside of the REPL, I call :gen-class to generate ui.exe.

Step 2. Instantiating the UI Objects

After I have loaded and required everything I need I can start creating objects. As you can see the creating objects is pretty straight forward, all I do is new up all the objects. In the next step is where the configuration of the objects and the event handlers are added.

Step 3. Setting Properties

The snippet above contains a sampling of the property setting code for the title label, the group box, the CheckedListBox and the 'Load Table' button. If you’d like to see the entire file you can do so here. You may notice all the calls to methods with names like set_Text and wonder where the methods come from since you do not see them when you look at the Label class in the Object Browser. In ClojureCLR we access the .NET objects at the CLR 'layer' which is different than the we access them through C#. The set_Text method is the CLR representation of the C# code Label.Text = "Some Text". If you’d like to see what other methods are available you can use the ildasm tool, which is a part of the .NET SDK. Here is an example of some of the set_ methods for the Label class:

Step 4. Adding the UI Objects to the Form and displaying the it

Now that I have all the objects customized I would like to show them to the user. In order to do that I need to add them to the Form object. The Form class has a Controls collection where all of its child objects are stored. I added the objects to the collection by calling the Controls.Add method for each object. When all of the objects have been added I call the ShowDialog method to display the UI.

Step 5. Adding Click Event Handlers to the Buttons.

A snazzy UI is great but if it doesn’t do anything what good is it? Handling the Button.Click events for both of the buttons is where all the action happens. In order to wire up the Button.Click event handlers I needed to use the gen-delegate macro. The macro’s signature looks like this:

gen-delegate
([type argVec & body])

The Load Table Click Event Handler

Here's what the 'Load Table' button's click handler looks like:

In my call to gen-delegate the parameters begin with passing the type System.EventHandler since we want a EventHandler delegate created. The next argument is the a vector which in this case contains the usual Click event parameters: sender and args. The last parameter is where the code of the event handler reside. In this case when the 'Load Table' button is clicked all of the table names from my bdb_post_2010 database are returned and are added to the CheckedListBox. After clicking the 'Load Table' button the app looks like this for me. Your table listing will look different.  The SQL File I provide only has three tables.

The tables are retrieved by creating a MySQL connection which is then passed to a function that retrieves the list of tables in the specified database.

The returned lazy sequence I passed through doseq so the tables can be added to the table list.  The table names are added by making a call to CheckedListBox.Items.Add method.  Once all of the table names have been added I close the database connection.

The Migrate Tables Event Handler

The ‘Migrate Tables…’ button’s event handler does a similar task as the ‘Load Table’ handler does, except it interacts with both MySQL and SQL Server databases. 

First it creates a connection to the MySQL database and uses the value from the 'New DB Name' TextBox to create the new database on the SQL Server instance. Next, doseq is called to process the values in the CheckedListBox.CheckedItems collection. Each of the selected table names is passed to mysql/get-columns to retrieve a sequence of maps that contains information about each column in the table. The returned sequence is then passed with the table name and SQL Server database object to sql/create-table which to creates the table in the SQL Server database. After all of the selected tables have been migrated to SQL Server the MySQL connection is closed and a MessageBox is displayed to inform the use that the migration is complete.

db/core.clj and db/sqlserver.clj

Since this is a blog post about creating a UI I will not be delving into the minute details of how the database related code works. However, I do think it is worth giving you a quick overview of the code. The DB code in db/mysql.clj, db/core.clj and db/sqlserver.clj are based on the code I wrote in Intro to Clojure-clr: Connecting to SQL Server and MySQL.

The db/core.clj file contains one function named run-sql which does what you might expect, it executes a query. In my previous database connectivity post I used (.Read reader) to loop through the results of a query but here I've switched to using the System.Data.DataTable class. Using the DataTable class allows me to close the reader right away and retrieve the column names for the query very easily. I used the column names with the zipmap function to map the column name to its values for each of the returned rows. This allows me to return a sequence of maps that represented the results of the query.

The db/mysql.clj file has three methods: get-connection, get-tables and get-columns. Each of these functions does what their names imply. get-tables uses 'show tables;' and get-columns uses 'describe [table name];'.

On the migration side of the app, the db/sqlserver.clj file makes use of the Microsoft.SqlServer.Smo assembly to create the new database, tables and columns. If you want more information on the visit the Microsoft.SqlServer.Management.Smo MSDN Page.

Running the App

Starting the application from the REPL is straight forward.  Start the REPL in the same directory that as the ui.clj file. When the REPL is ready enter:

You should then see the UI appear with an empty CheckListBox.  Once the app is up and running click on the ‘Load Table’ button.  If are used the SQL file I mentioned above you should see three tables:  master, schools, and schoolsplayers.

Go ahead and click all three tables.  You will need to click on the table once to highlight it and then either click on the check box or press the space bar to put a check in the box.

Next, enter the new database name.  My database will be named clr-intro-4. Finally, I kick off the migration click on the ‘Migrate Tables…’ button and within a few seconds you will have a new database with the three tables added. When the migration has completed you will see something similar to the image below.

Summary

Building a UI with System.Windows.Forms in ClojureCLR is straight forward. You only need to bring in the System.Windows.Forms assembly and follow the typical process of creating a Forms based application.

Adding event handlers is as easy as making use of the gen-delegate macro passing it the ‘guts’ of what you want to do when a particular event is fired.

About the Data

The data I’m using for this blog post comes from the Baseball Databank project.  The project has gathered all the baseball status from previous season and offers the data in many different formats.

Resources

Previous Posts: Getting Started with Clojure-clr, Intro to Clojure-clr: Connecting to SQL Server and MySQL

Database Downloads: SQL Server Express, MySQL

Data: mysql_schema.sql

My Source: https://github.com/rippinrobr/clojure-clr-intro/zipball/master

More Examples and Information: Another Windows Forms Example, Microsoft.SqlServer.Management.Smo MSDN Page, and Ildasm.exe page on MSDN