Announcing a New ClojureCLR Community

Not a huge post today but I did want to pass along that David Miller has created the ClojureCLR Google+ community to "share information about working with ClojureCLR". Join the community, ask questions, let everyone know what you are working on or just catch up on ClojureCLR news. Join today and let us all know what you are working on.

Intro to ClojureScript - Part 3 - Using Shoreleave-pubsub

In this post we will move away from the function based event handling we put in place in my previous post to use Shoreleave’s pubsub library. Using pubsub will allow use to easily add and remove functionality over the lifetime of our app. Once we’ve replaced the current tightly coupled event handling we will add console logging and the ability to view our search history.

The pubsub library works like the name implies, topics are published and can be subscribed to. Topics can be strings, keywords, functions, atoms, web workers or local storage. In this post we will use keywords to indicate when a search is to be performed, when the results come back, and when the view history button is clicked. An atom will be used to track the current search and keep a list of previous searches.

Since ClojureScript is a bit of a mouthful I will use the abbreviation cljs for the remainder of this post. Remember I am learning cljs as I create these posts so any and all guidance is appreciated.

The Goal

By the end of this post we will have converted the event handling code over to use shorelave-pubsub and have added console logging and the ability to view our search history for the current session.

The Setup

If you don't already have cljs installed on your system go read Intro to ClojureScript - Getting Started to get your environment ready for cljs development. You will also need the code from my previous post. If you don't already have it you can grab it from my cljs-intro project on the part-2 branch. Once your environment is setup we can start making the necessary changes to add pubsub support.

Project.clj Changes

We only have one change to make in the project.clj file and that is to add the reference to the shoreleave-pubsub library:

[shoreleave/shoreleave-pubsub "0.2.2"]

Once the change is in place save the file and run lein deps.

Index.html Changes

We need to add the 'View Search History' button to our web page so users can view their previous searches. Add the new button next to the ‘Get Stats!’ button. The updated index.html should look like this:

After updating the HTML we are ready to get into the cljs changes. The first step is to create the new file that will contain the pubsub code.

The PubSub Code

A little background

Before we get into the pubsub code lets take a minute to go over shoreleave-pubsub. The shoreleave-pubsub library uses two protocols IMessageBrokerBus and IPublishable to provide the necessary functionality. The IMessageBrokerBus manages the bus, giving us the ability to publish, subscribe and unsubscribe to a given bus. The IPublishable protocol provides us with functions that define things that can be published. We will use the topicify and publishize functions to convert the keywords and atom to topics. For a more detailed look into what these two protocols provide read the Shoreleave pubsub docs.

src/cljs/pubsub.cljs

As you might expect the pubsub.cljs file is where we’ll put our pubsub related code will live. The file is broken down into three areas: config, helper functions and subscriptions.

config section

The first thing we need to do is require two namespaces shoreleave.pubsub.simple and shoreleave.pubsub.protocols. shoreleave.pubsubs.simple provides us with a synchronous bus which is fine for this example. The shoreleave.pubsubs.protocols library provides the functions we need to turn our keywords into topics and make our search-state atom ‘publishable’.

First we create the bus and then we ‘topicify’ the three keywords, :search, :results, and :history. Once the keywords have been 'topicify-ed' we can use them to publish messages. The search-topic will be used anytime a user clicks the ‘Get Stats!’ button. The results-topic will be used when that stats are returned. As you might expect the history-topic will be used to indicate that the user has clicked the ‘View Search History’ button.

Turning the search-state atom into something that can be used as a topic is slightly different. We need to call the publishize function passing in the atom and the bus it will be published on. Calling the publishize adds a watch function that publishes all changes to the atom’s subscribers. The published data contains a copy of the new and the old state of the atom. Subscribers can access the data by using the either the :new or :old keyword.

helper functions

The helper functions are there to keep the ‘guts’ of the pubsub code out of the search.cljs file. The publish-* functions are there so code in the search.cljs can publish messages to all subscribers. The subscribe-to function wraps the pubsub subscription process so subscriptions can be created outside of the pubsub.cljs.

subscriptions

The last two lines of the file create subscriptions to search-topic and search-state. These subscriptions, subscribe the anonymous functions to each topic. Now, anytime there is a message published on either one of these topics the published data will be written out to console.log.

src/cljs/search.cljs

The search.cljs file underwent quite a few changes during the conversion. Anything to do with event handling changed as did some of the functions called from within the event handlers. In addition to coding changes we had to add a new require statement for the new namespace cljs-intro.pubsub.

Subscriptions and Event handling

Since we are using pubsub to handle events and state changes we need subscribe to the topics we are interested in.

The subscription calls are using the helper function subscribe-to which takes the topic you want to subscribe to and the function that will be called when a new message is published.

Now when a handler is called instead of calling the function to handle the event, a message is publish for a specific topic. For instance, when a user clicks the ‘Get Stats!’ the handler publishes a message on the topic search-topic passing the value that was entered by the user to all subscribers. Here’s the difference between the two versions:

When we subscribed to the search-topic topic we established that the player-lookup function would be called. There was only two changes made, the first is to publish the results of the remote call when it returns instead of calling display-results directly. The published data is converted to a Clojure data structure prior to publishing freeing the subscribers of any unnecessary steps prior to using the data.

The display-results function is subscribed to the results-topic. The new display-results receives the converted data so we no longer have to convert it before we display it. I’ve also created a new function, update-results-div, to handle the updating of the search results.

The functionality we just walked through is the same as it was before we switched to using pubsub. You may be wondering whats the point? The conversion to pubsub allows us to open up the application so that adding new functionality can be done with little to no changes in the search.cljs code. As an example, we added console logging for the search string by adding a subscription to the search-topic and giving it a function that calls js/console.log. No changes were required to the core search code. The image above shows the search logging and the logging that occurs anytime the search-state atom changes. Thank you pubsub.

Viewing Search History

Another piece of functionality that was added is the ability to see the previous searches you’ve run.

When does search-state change? Every time a new search is submitted the current value of :lastname is conj’d into the :previous-searches sequence provided that :lastname is not nil or empty. When that happens a new message is published to notify all subscribers of the change. In our app we have subscriptions to search-state for managing the visibility of the ‘View Search History’ button and for logging.

The Code

When search-state is changed the search-state-change function is called. This function is where the visibility of the ‘View Search History’ button is determined. If the :previous-searches sequence contains at least one string then the domina set-styles! function is called to make the button visible.

The history button’s click event handler publishes a message on the history-topic. Our subscription for this topic calls cljs-intro.views/view-history function which takes the :previous-searches sequence as its only parameter. From that sequence an unordered HTML list is created and displayed using the new function update-results-div. Here’s what it looks like after a few searches and a click of the ‘View Search History’ button.



Summary

We now have an app that uses shoreleave-pubsub for handling events instead of the traditional, tightly coupled event handler functions. Using the library has allowed us to add new search related functionality without making any changes to the core part of our application. Now we have a more flexible application.

Whats Next?

In Intro to ClojureScript - Part 4 - Using Shoreleave’s Remoting we will convert the current remote callto the server to use shoreleave-remoting. Part 4 may not be up until two weeks due to an overbooked schedule. I will be speaking at Richmond Code Camp giving an ‘Intro to ClojureCLR’ talk and I need to get started on my talk. As soon as thats done Part 4 will be up.

Resources

shoreleave-pubsub docs. I spent a lot of time reading the docs for shoreleave-pubsub.  

Hockey Databank Database -- I created the database using data that originates from a Yahoo! group called hockey databank. After each season the group produces updated CSV files with the stats of the previous NHL season in addition to previous years data.

ClojureScript Experience Report - Resources Bits of information about Jason’s ClojureScript based application and his experience with ClojureScript.

ClojureScript: Up and Running - An early release book that discusses ClojureScript. I’ve read the released chapters and found it to be a good resource. Chapters 2 and 3 go over ClojureScript’s compilation, project structure, and other informative tidbits for both ClojureScript and Clojure. I’ve recommended the book to a colleague who is learning Clojure because of the way the authors describe data structures, immutability and sequences.

cljs-intro - My github repo that houses the code for this blog series. Each part of the series has its own branch. This post’s code is on the Part 3 branch.




Intro to ClojureScript - Part 2 - Getting the Stats

In this post we will flesh out the backend of the app we started in Intro to ClojureScript - Getting Started. We are going to create a web service to lookup player stats by last name. Once we have the lookup service in place we will update search.cljs to fetch the results of our query using the Google closure library’s goog.net.XhrIo. When the data is returned we will display it to the user using the hiccups library.

Since ClojureScript is a bit of a mouthful I will use the abbreviation cljs for the remainder of this post. Remember I am learning cljs as I create these posts so any and all guidance is appreciated.

The Goal

By the end of this post we will have a web based app that will lookup player stats by last name, displaying the results of our search.

The Setup

If you don’t already have cljs setup on your machine read my Getting Started post. It will walk you through the necessary steps to set up your cljs environment. I am also assuming you have the code from the first post. If not you can grab it from my cljs-intro github repo.

Grabbing the DB

We will be using a SQLite version of my hockey stats database. You can download it from my hockeydb github project. Once you have the database, create a db directory in the project’s home directory and copy the downloaded database into it.

Project.clj Changes

Since we will be interacting with a database, manipulating the page’s DOM, and creating a web ‘service’ we need to add a the following dependencies to our project:

[org.clojure/java.jdbc “0.2.3”]
[sqlitejdbc “0.5.6”]                  
[domina “1.0.0”]
[noir “1.3.0-beta0”]
[hiccups “0.1.1”]

The first two libraries are for our database interaction. The domina library allows us to manipulate the DOM from our cljs code. We will use noir to create our lookup web service. The hiccups library will be used to add and remove HTML from the web page.

The last setup step is to update the resources/public/index.html file. Most of the changes in it are for twitter bootstrap support. Adding bootstrap will make the search results a little easer to read.

The only change needed for the application is the addition of results div. As you might of guessed this is where the div that will contain the search results.

Creating the DB Access Code

All of our database related code will live in src/clj/cljs_intro/db.clj file. The code does a few simple queries to gather the player related information. I’m not going to go through this code in detail since it is ‘normal’ Clojure and not cljs. If you'd like to see the code here's the link to at github: db.clj.

In the db.clj file I’ve made the executive decision to return the stats for the first matching player. Why? I don’t wan’t to spend a bunch of time on non-cljs code so I made it simple on myself. What does that mean? It means if you search for Gretzky you’ll get Brett Gretzky instead of Wayne. I bet you didn't know there was a Brett Gretzky did ya.

Creating ‘Web Server’

Now that we have the database its time to create our simple web server. Noir provides functions that allow us to respond to HTTP requests, mapping routes to code and return JSON data. Our server will listen on the port 8888 for following two URLs:

http://localhost:8888/index.html
http://localhost:8888/player/:lastname.

The last URL has a parameter (:lastname) which will be the player’s last name entered by the user. Any other request will throw a Noir error.

Now that our server is complete we need to update the project.clj file so we can start it by running `lein run`. Add the following to the project.clj file:

:main cljs-intro.core

Lets test the server out. At the command prompt enter:

lein run

When the server starts up open your favorite browser and go to http://localhost:8888/player/ricci . If everything goes well you should see about a page full of JSON data.

Retrieving and Displaying the Data

If you go to http://localhost:8888/index.html and enter ricci in the text box and click the ‘Get Stats!’ button you’ll see an alert box with ricci in it. That was fine for part 1 but now we are ready to get the stats. In order to do that we need to update the ‘Get Stats!’ click event handler. Instead of a js/alert call we will call our player-lookup function.

The Updated Click Event Handler

The player-lookup function wraps a call to the Google Closure library’s goog.net.XhrIo.send method.

The goog.net.XhrIo.send function has two parameters, the URL to call and a callback function to process the returned data. We create the URL parameter by concatenating the base URL (/player) with the user’s input, in our example its ricci. When the call returns the data is passed to the display-results function so we can, well display the results.

display-results

The display-results converts the returned JSON data into a Clojure data structure and manages the UI updates.

The call to js->clj coverts the JSON to a Clojure data structure. A key thing to note here are the parameters :keywordize-keys true. Before I used those two parameters I was having a hell of a time getting to the data. After using the parameters I could easily access the data using keywords. I’m assuming that most of the time you use js->clj you’ll want to add :keywordize-keys true.

The next line uses the domina.xpath/xpath function to get a reference to the results div DOM object. We will use the reference to display the search results.

Before displaying the results of the current query we remove any previous query results by using the domina function called destroy-children!. The parameter for the destroy-children! function is a reference to a DOM object with the children to be removed. Now that the display area is empty we can display the results of the new search. On the last line a call is made to cljs-intro.views/show-stats which is where the HTML generation process starts. The results of that call are appended to the results div by calling domina’s append! function.

The HTML

Creating the HTML is done using the hiccups library which is a port of the Clojure hiccup library. Since I've used hiccup and noir for non-cljs web-based apps using hiccups made the HTML generation easy. The only difference I saw between hiccups and hiccup was that I had to use defhtml when creating a function that generated HTML.

If you wish to view the code that generates the email you can see the views.cljs. Now, when we run the search for Ricci we see the following page displayed:



Summary

The search app makes a remote call using the Google Closure library. When the data returned we were able to convert the JSON to a Clojure data structure. Once the conversion was complete we used the domina library to remove and add HTML with help from the hiccups library. Our application actually returns the stats for the player (as long as you only want the first person with the last name in question).

Whats Next?

In Intro to ClojureScript - Part 3 - Using Shoreleave - we will use the Shoreleave libraries to decouple the front end from the back end by using event based messaging. We will also change the way we handle the remote calls to use the shoreleave-remote library.

Resources

twitterbuzz and this stack overflow post implementing an ajax call in clojurescript helped me with the remote call.

ClojureScriptOne.com - used for DOM manipulation guidance.

Hockey Databank Database -- I created the database using data that originates from a Yahoo! group called hockey databank. After each season the group produces updated CSV files with the stats of the previous NHL season in addition to previous years data.

ClojureScript Experience Report - Resources Bits of information about Jason’s ClojureScript based application and his experience with ClojureScript.

ClojureScript: Up and Running - An early release book that discusses ClojureScript. I’ve read the released chapters and found it to be a good resource. Chapters 2 and 3 go over ClojureScript’s compilation, project structure, and other informative tidbits for both ClojureScript and Clojure. I’ve recommended the book to a colleague who is learning Clojure because of the way the authors describe data structures, immutability and sequences.

cljs-intro - My github repo that stores the code for this blog series. The code for Part 1 can be found on the branch Part 1. This post’s code is on the Part 2 branch.