tag:blogger.com,1999:blog-7249938538643087942024-03-18T19:00:40.430-04:00My Clojure AdventureFollow me as I delve into the world of Clojure-clr and Clojure from a .NET developer's prospective.Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.comBlogger37125tag:blogger.com,1999:blog-724993853864308794.post-44291843860889225832013-01-22T19:18:00.000-05:002013-01-22T19:19:53.772-05:00Announcing a New ClojureCLR Community<p>Not a huge post today but I did want to pass along that David Miller has created the <a href="https://plus.google.com/u/0/communities/106235975067046753407">ClojureCLR Google+ community</a> to "share information about working with ClojureCLR". Join the community, ask questions, let everyone know what you are working on or just catch up on ClojureCLR news. Join today and let us all know what you are working on.</p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com8tag:blogger.com,1999:blog-724993853864308794.post-62232298242083320952012-09-19T10:22:00.002-04:002012-09-19T10:26:42.868-04:00Intro to ClojureScript - Part 3 - Using Shoreleave-pubsub<p>
In this post we will move away from the function based event handling we put in place in <a href="http://www.myclojureadventure.com/2012/09/intro-to-clojurescript-part-2-getting.html">my previous post</a> to use <a href="https://github.com/shoreleave/shoreleave-pubsub">Shoreleave’s pubsub library</a>. Using pubsub will allow use to easily add and remove functionality over the lifetime of our app. Once we’ve replaced the current tightly coupled event handling we will add console logging and the ability to view our search history.
</p>
<p>
The pubsub library works like the name implies, topics are published and can be subscribed to. Topics can be strings, keywords, functions, atoms, web workers or local storage. In this post we will use keywords to indicate when a search is to be performed, when the results come back, and when the view history button is clicked. An atom will be used to track the current search and keep a list of previous searches.
</p>
<p><i>
Since ClojureScript is a bit of a mouthful I will use the abbreviation cljs for the remainder of this post. Remember I am learning cljs as I create these posts so any and all guidance is appreciated.
</i></p>
<h3>The Goal</h3>
<p>
By the end of this post we will have converted the event handling code over to use shorelave-pubsub and have added console logging and the ability to view our search history for the current session.
</p>
<h3>The Setup</h3>
<p>
If you don't already have cljs installed on your system go read <a href="http://www.myclojureadventure.com/2012/09/intro-to-clojurescript-getting-started.html">Intro to ClojureScript - Getting Started</a> to get your environment ready for cljs development. You will also need the code from my previous post. If you don't already have it you can grab it from my <a href="https://github.com/rippinrobr/cljs-intro/tree/part-2">cljs-intro project on the part-2 branch</a>. Once your environment is setup we can start making the necessary changes to add pubsub support.
</p>
<h2>Project.clj Changes</h2>
<p>
We only have one change to make in the project.clj file and that is to add the reference to the shoreleave-pubsub library:
</p>
<pre>
[shoreleave/shoreleave-pubsub "0.2.2"]
</pre>
<p>
Once the change is in place save the file and run lein deps.
</p>
<h2>Index.html Changes</h2>
<p>
We need to add the 'View Search History' button to our web page so users can view their previous searches. Add the new button next to the ‘Get Stats!’ button. The updated index.html should look like this:
</p>
<script src="https://gist.github.com/3739309.js?file=gistfile1.html"></script>
<p>
After updating the HTML we are ready to get into the cljs changes. The first step is to create the new file that will contain the pubsub code.
</p>
<h3>The PubSub Code</h3>
<h2>A little background</h2>
<p>
Before we get into the pubsub code lets take a minute to go over shoreleave-pubsub. The shoreleave-pubsub library uses two protocols IMessageBrokerBus and IPublishable to provide the necessary functionality. The IMessageBrokerBus manages the bus, giving us the ability to publish, subscribe and unsubscribe to a given bus. The IPublishable protocol provides us with functions that define things that can be published. We will use the topicify and publishize functions to convert the keywords and atom to topics. For a more detailed look into what these two protocols provide read the <a href="http://shoreleave.github.com/shoreleave-pubsub/#shoreleave.pubsubs.protocols">Shoreleave pubsub docs</a>.
</p>
<h2>src/cljs/pubsub.cljs</h2>
<p>
As you might expect the pubsub.cljs file is where we’ll put our pubsub related code will live. The file is broken down into three areas: config, helper functions and subscriptions.
</p>
<script src="https://gist.github.com/3743213.js?file=gistfile1.clj"></script>
<h2>config section</h2>
<p>
The first thing we need to do is require two namespaces shoreleave.pubsub.simple and shoreleave.pubsub.protocols. shoreleave.pubsubs.simple provides us with a synchronous bus which is fine for this example. The shoreleave.pubsubs.protocols library provides the functions we need to turn our keywords into topics and make our search-state atom ‘publishable’.
</p>
<p>
First we create the bus and then we ‘topicify’ the three keywords, :search, :results, and :history. Once the keywords have been 'topicify-ed' we can use them to publish messages. The search-topic will be used anytime a user clicks the ‘Get Stats!’ button. The results-topic will be used when that stats are returned. As you might expect the history-topic will be used to indicate that the user has clicked the ‘View Search History’ button.
</p>
<p>
Turning the search-state atom into something that can be used as a topic is slightly different. We need to call the publishize function passing in the atom and the bus it will be published on. Calling the publishize adds a watch function that publishes all changes to the atom’s subscribers. The published data contains a copy of the new and the old state of the atom. Subscribers can access the data by using the either the :new or :old keyword.
</p>
<script src="https://gist.github.com/3743337.js?file=gistfile1.clj"></script>
<h2>helper functions</h2>
<p>
The helper functions are there to keep the ‘guts’ of the pubsub code out of the search.cljs file. The publish-* functions are there so code in the search.cljs can publish messages to all subscribers. The subscribe-to function wraps the pubsub subscription process so subscriptions can be created outside of the pubsub.cljs.
</p>
<script src="https://gist.github.com/3743398.js?file=gistfile1.clj"></script>
<h2>subscriptions</h2>
<p>
The last two lines of the file create subscriptions to search-topic and search-state. These subscriptions, subscribe the anonymous functions to each topic. Now, anytime there is a message published on either one of these topics the published data will be written out to console.log.
</p>
<h2>src/cljs/search.cljs</h2>
<p>
The search.cljs file underwent quite a few changes during the conversion. Anything to do with event handling changed as did some of the functions called from within the event handlers. In addition to coding changes we had to add a new require statement for the new namespace cljs-intro.pubsub.
</p>
<h3>Subscriptions and Event handling </h3>
<p>
Since we are using pubsub to handle events and state changes we need subscribe to the topics we are interested in.
</p>
<script src="https://gist.github.com/3743510.js?file=gistfile1.clj"></script>
<p>
The subscription calls are using the helper function subscribe-to which takes the topic you want to subscribe to and the function that will be called when a new message is published.
</p>
<p>
Now when a handler is called instead of calling the function to handle the event, a message is publish for a specific topic. For instance, when a user clicks the ‘Get Stats!’ the handler publishes a message on the topic search-topic passing the value that was entered by the user to all subscribers. Here’s the difference between the two versions:
</p>
<script src="https://gist.github.com/3745508.js?file=gistfile1.clj"></script>
<p>
When we subscribed to the search-topic topic we established that the player-lookup function would be called. There was only two changes made, the first is to publish the results of the remote call when it returns instead of calling display-results directly. The published data is converted to a Clojure data structure prior to publishing freeing the subscribers of any unnecessary steps prior to using the data.
</p>
<script src="https://gist.github.com/3745626.js?file=gistfile1.clj"></script>
<p>
The display-results function is subscribed to the results-topic. The new display-results receives the converted data so we no longer have to convert it before we display it. I’ve also created a new function, update-results-div, to handle the updating of the search results.
</p>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-UMabPm20M_M/UFnN_3_TUAI/AAAAAAAAAt0/H6uBBz59ZS0/s1600/console.png" imageanchor="1" style="margin-left:1em; margin-right:1em"><img border="0" height="311" width="484" src="http://2.bp.blogspot.com/-UMabPm20M_M/UFnN_3_TUAI/AAAAAAAAAt0/H6uBBz59ZS0/s320/console.png" /></a></div>
<p>
The functionality we just walked through is the same as it was before we switched to using pubsub. You may be wondering whats the point? The conversion to pubsub allows us to open up the application so that adding new functionality can be done with little to no changes in the search.cljs code. As an example, we added console logging for the search string by adding a subscription to the search-topic and giving it a function that calls js/console.log. No changes were required to the core search code. The image above shows the search logging and the logging that occurs anytime the search-state atom changes. Thank you pubsub.
</p>
<h3>Viewing Search History</h3>
<p>
Another piece of functionality that was added is the ability to see the previous searches you’ve run.
</p>
<script src="https://gist.github.com/3745728.js?file=gistfile1.clj"></script>
<p>
When does search-state change? Every time a new search is submitted the current value of :lastname is conj’d into the :previous-searches sequence provided that :lastname is not nil or empty. When that happens a new message is published to notify all subscribers of the change. In our app we have subscriptions to search-state for managing the visibility of the ‘View Search History’ button and for logging.
</p>
<h2>The Code</h2>
<script src="https://gist.github.com/3745886.js?file=gistfile1.clj"></script>
<p>
When search-state is changed the search-state-change function is called. This function is where the visibility of the ‘View Search History’ button is determined. If the :previous-searches sequence contains at least one string then the domina set-styles! function is called to make the button visible.
</p>
<p>
The history button’s click event handler publishes a message on the history-topic. Our subscription for this topic calls cljs-intro.views/view-history function which takes the :previous-searches sequence as its only parameter. From that sequence an unordered HTML list is created and displayed using the new function update-results-div. Here’s what it looks like after a few searches and a click of the ‘View Search History’ button.
</p>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-M4pWGtnVEAI/UFnOtKCVeVI/AAAAAAAAAuA/W8-E_U-_a-4/s1600/view-history.png" imageanchor="1" style="margin-left:1em; margin-right:1em"><img border="0" height="143" width="400" src="http://1.bp.blogspot.com/-M4pWGtnVEAI/UFnOtKCVeVI/AAAAAAAAAuA/W8-E_U-_a-4/s400/view-history.png" /></a></div>

<h3>Summary</h3>
<p>
We now have an app that uses shoreleave-pubsub for handling events instead of the traditional, tightly coupled event handler functions. Using the library has allowed us to add new search related functionality without making any changes to the core part of our application. Now we have a more flexible application.
</p>
<h3>Whats Next?</h3>
<p>
In Intro to ClojureScript - Part 4 - Using Shoreleave’s Remoting we will convert the current remote callto the server to use shoreleave-remoting. Part 4 may not be up until two weeks due to an overbooked schedule. I will be speaking at <a href="http://richmondcodecamp.org/">Richmond Code Camp</a> giving an ‘Intro to ClojureCLR’ talk and I need to get started on my talk. As soon as thats done Part 4 will be up.
</p>
<h3>Resources</h3>
<p>
<a href="http://shoreleave.github.com/shoreleave-pubsub/#shoreleave.pubsubs.protocols">shoreleave-pubsub docs</a>. I spent a lot of time reading the docs for shoreleave-pubsub.
</p>
<p>
<p>
<a href="http://progadventure.blogspot.com/2012/07/hockey-databank-databases.html">Hockey Databank Database</a> -- I created the database using data that originates from a Yahoo! group called <a href="http://sports.groups.yahoo.com/group/hockey-databank/">hockey databank</a>. After each season the group produces updated CSV files with the stats of the previous NHL season in addition to previous years data.
</p>
<p>
<a href="http://jasonrudolph.com/blog/2012/08/17/clojurescript-experience-report-resources/">ClojureScript Experience Report - Resources</a> Bits of information about Jason’s ClojureScript based application and his experience with ClojureScript.
</p>
<p>
<a href="http://shop.oreilly.com/product/0636920025139.do">ClojureScript: Up and Running</a> - An early release book that discusses ClojureScript. I’ve read the released chapters and found it to be a good resource. Chapters 2 and 3 go over ClojureScript’s compilation, project structure, and other informative tidbits for both ClojureScript and Clojure. I’ve recommended the book to a colleague who is learning Clojure because of the way the authors describe data structures, immutability and sequences.
</p>
<p>
<a href="https://github.com/rippinrobr/cljs-intro">cljs-intro</a> - My github repo that houses the code for this blog series. Each part of the series has its own branch. This post’s code is on the <a href="https://github.com/rippinrobr/cljs-intro/tree/part-3">Part 3</a> branch.
</p>
Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com4tag:blogger.com,1999:blog-724993853864308794.post-9440809333509514182012-09-12T08:41:00.000-04:002012-09-12T08:41:29.487-04:00Intro to ClojureScript - Part 2 - Getting the Stats <p>
In this post we will flesh out the backend of the app we started in <a href="http://www.myclojureadventure.com/2012/09/intro-to-clojurescript-getting-started.html">Intro to ClojureScript - Getting Started</a>. We are going to create a web service to lookup player stats by last name. Once we have the lookup service in place we will update search.cljs to fetch the results of our query using the Google closure library’s <a href="http://closure-library.googlecode.com/svn/docs/class_goog_net_XhrIo.html">goog.net.XhrIo</a>. When the data is returned we will display it to the user using the <a href="https://github.com/teropa/hiccups">hiccups library</a>.
</p>
<p><i>
Since ClojureScript is a bit of a mouthful I will use the abbreviation cljs for the remainder of this post. Remember I am learning cljs as I create these posts so any and all guidance is appreciated.
</i></p>
<h3>The Goal</h3>
<p>
By the end of this post we will have a web based app that will lookup player stats by last name, displaying the results of our search.
</p>
<h3>The Setup</h3>
<p>
If you don’t already have cljs setup on your machine read my <a href="http://www.myclojureadventure.com/2012/09/intro-to-clojurescript-getting-started.html">Getting Started post</a>. It will walk you through the necessary steps to set up your cljs environment. I am also assuming you have the code from the first post. If not you can grab it from my <a href="https://github.com/rippinrobr/cljs-intro/tree/part-1">cljs-intro github repo</a>.
</p>
<h2>Grabbing the DB</h2>
<p>We will be using a SQLite version of my hockey stats database. You can download it from my <a href="https://github.com/downloads/rippinrobr/hockeydb/hockeydb-6-23-12-sqlite.tgz">hockeydb github</a> project. Once you have the database, create a db directory in the project’s home directory and copy the downloaded database into it.
<h2>Project.clj Changes</h2>
<p>
Since we will be interacting with a database, manipulating the page’s DOM, and creating a web ‘service’ we need to add a the following dependencies to our project:
</p>
<pre>
[org.clojure/java.jdbc “0.2.3”]
[sqlitejdbc “0.5.6”]
[domina “1.0.0”]
[noir “1.3.0-beta0”]
[hiccups “0.1.1”]
</pre>
<p>
The first two libraries are for our database interaction. The domina library allows us to manipulate the DOM from our cljs code. We will use noir to create our lookup web service. The hiccups library will be used to add and remove HTML from the web page.
</p>
<p>
The last setup step is to update the resources/public/index.html file. Most of the changes in it are for twitter bootstrap support. Adding bootstrap will make the search results a little easer to read.
</p>
<script src="https://gist.github.com/3668949.js?file=index.html"></script>
<p>
The only change needed for the application is the addition of results div. As you might of guessed this is where the div that will contain the search results.
</p>
<h3>Creating the DB Access Code</h3>
<p>
All of our database related code will live in src/clj/cljs_intro/db.clj file. The code does a few simple queries to gather the player related information. I’m not going to go through this code in detail since it is ‘normal’ Clojure and not cljs. If you'd like to see the code here's the link to at github: <a href="https://github.com/rippinrobr/cljs-intro/blob/part-2/src/clj/cljs_intro/db.clj">db.clj</a>.
</p>
<p><i>
In the db.clj file I’ve made the executive decision to return the stats for the first matching player. Why? I don’t wan’t to spend a bunch of time on non-cljs code so I made it simple on myself. What does that mean? It means if you search for Gretzky you’ll get Brett Gretzky instead of Wayne. I bet you didn't know there was a Brett Gretzky did ya.
</i></p>
<h3>Creating ‘Web Server’</h3>
<p>
Now that we have the database its time to create our simple web server. Noir provides functions that allow us to respond to HTTP requests, mapping routes to code and return JSON data. Our server will listen on the port 8888 for following two URLs:
</p>
<p>
http://localhost:8888/index.html <br/>
http://localhost:8888/player/:lastname.
</p>
<p>
The last URL has a parameter (:lastname) which will be the player’s last name entered by the user. Any other request will throw a Noir error.
</p>
<script src="https://gist.github.com/3668765.js?file=gistfile1.clj"></script>
<p>
Now that our server is complete we need to update the project.clj file so we can start it by running `lein run`. Add the following to the project.clj file:
</p>
<pre>
:main cljs-intro.core
</pre>
<p>
Lets test the server out. At the command prompt enter:
</p>
<pre>
lein run
</pre>
<p>
When the server starts up open your favorite browser and go to <a href="http://localhost:8888/player/ricci">http://localhost:8888/player/ricci</a> . If everything goes well you should see about a page full of JSON data.
</p>
<h3>Retrieving and Displaying the Data</h3>
<p>
If you go to <a href="http://localhost:8888/index.html">http://localhost:8888/index.html</a> and enter ricci in the text box and click the ‘Get Stats!’ button you’ll see an alert box with ricci in it. That was fine for part 1 but now we are ready to get the stats. In order to do that we need to update the ‘Get Stats!’ click event handler. Instead of a js/alert call we will call our player-lookup function.
</p>
<h2>
The Updated Click Event Handler
</h2>
<p>
The player-lookup function wraps a call to the Google Closure library’s goog.net.XhrIo.send method.
</p>
<script src="https://gist.github.com/3669064.js?file=gistfile1.clj"></script>
<p>
The goog.net.XhrIo.send function has two parameters, the URL to call and a callback function to process the returned data. We create the URL parameter by concatenating the base URL (/player) with the user’s input, in our example its ricci. When the call returns the data is passed to the display-results function so we can, well display the results.
</p>
<h2>display-results</h2>
<p>
The display-results converts the returned JSON data into a Clojure data structure and manages the UI updates.
</p>
<script src="https://gist.github.com/3669174.js?file=gistfile1.clj"></script>
<p>
The call to js->clj coverts the JSON to a Clojure data structure. A key thing to note here are the parameters :keywordize-keys true. Before I used those two parameters I was having a hell of a time getting to the data. After using the parameters I could easily access the data using keywords. I’m assuming that most of the time you use js->clj you’ll want to add :keywordize-keys true.
</p>
<p>
The next line uses the domina.xpath/xpath function to get a reference to the results div DOM object. We will use the reference to display the search results.
</p>
<p>
Before displaying the results of the current query we remove any previous query results by using the domina function called destroy-children!. The parameter for the destroy-children! function is a reference to a DOM object with the children to be removed. Now that the display area is empty we can display the results of the new search. On the last line a call is made to cljs-intro.views/show-stats which is where the HTML generation process starts. The results of that call are appended to the results div by calling domina’s append! function.
</p>
<h2>The HTML</h2>
<p>
Creating the HTML is done using the hiccups library which is a port of the Clojure hiccup library. Since I've used hiccup and noir for non-cljs web-based apps using hiccups made the HTML generation easy. The only difference I saw between hiccups and hiccup was that I had to use defhtml when creating a function that generated HTML.
</p>
<p>
If you wish to view the code that generates the email you can see the <a href="https://github.com/rippinrobr/cljs-intro/blob/part-2/src/cljs/views.cljs"> views.cljs</a>. Now, when we run the search for Ricci we see the following page displayed:
</p>
<div class="separator" style="clear: both; text-align: left;">
<a href="http://2.bp.blogspot.com/-SfZKfMwMUHE/UE-V4MbonYI/AAAAAAAAAtg/RXdeeXT55GY/s1600/ricci_results.png" imageanchor="1" style="margin-left:1em; margin-right:1em"><img border="0" height="268" width="500" src="http://2.bp.blogspot.com/-SfZKfMwMUHE/UE-V4MbonYI/AAAAAAAAAtg/RXdeeXT55GY/s320/ricci_results.png" /></a></div>
<h3>Summary</h3>
<p>
The search app makes a remote call using the Google Closure library. When the data returned we were able to convert the JSON to a Clojure data structure. Once the conversion was complete we used the domina library to remove and add HTML with help from the hiccups library. Our application actually returns the stats for the player (as long as you only want the first person with the last name in question).
</p>
<h3>Whats Next?</h3>
<p>
In <i>Intro to ClojureScript - Part 3 - Using Shoreleave </i>- we will use the Shoreleave libraries to decouple the front end from the back end by using event based messaging. We will also change the way we handle the remote calls to use the shoreleave-remote library. </p>
<h3>Resources</h3>
<p>
<a href="https://github.com/clojure/clojurescript/tree/master/samples/twitterbuzz">twitterbuzz</a> and this stack overflow post <a href="http://stackoverflow.com/questions/8554745/implementing-an-ajax-call-in-clojurescript">implementing an ajax call in clojurescript</a> helped me with the remote call.
</p>
<p>
<a href="http://clojurescriptone.com">ClojureScriptOne.com</a> - used for DOM manipulation guidance.
</p>
<p>
<a href="http://progadventure.blogspot.com/2012/07/hockey-databank-databases.html">Hockey Databank Database</a> -- I created the database using data that originates from a Yahoo! group called <a href="http://sports.groups.yahoo.com/group/hockey-databank/">hockey databank</a>. After each season the group produces updated CSV files with the stats of the previous NHL season in addition to previous years data.
</p>
<p>
<a href="http://jasonrudolph.com/blog/2012/08/17/clojurescript-experience-report-resources/">ClojureScript Experience Report - Resources</a> Bits of information about Jason’s ClojureScript based application and his experience with ClojureScript.
</p>
<p>
<a href="http://shop.oreilly.com/product/0636920025139.do">ClojureScript: Up and Running</a> - An early release book that discusses ClojureScript. I’ve read the released chapters and found it to be a good resource. Chapters 2 and 3 go over ClojureScript’s compilation, project structure, and other informative tidbits for both ClojureScript and Clojure. I’ve recommended the book to a colleague who is learning Clojure because of the way the authors describe data structures, immutability and sequences.
</p>
<p>
<a href="https://github.com/rippinrobr/cljs-intro">cljs-intro</a> - My github repo that stores the code for this blog series. The code for <a href="https://github.com/rippinrobr/cljs-intro/tree/part-1">Part 1</a> can be found on the branch <a href="https://github.com/rippinrobr/cljs-intro/tree/part-1">Part 1</a>. This post’s code is on the <a href="https://github.com/rippinrobr/cljs-intro/tree/part-2">Part 2</a> branch.
</p>
Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com324tag:blogger.com,1999:blog-724993853864308794.post-83562083280842866322012-09-04T11:11:00.000-04:002012-09-04T11:13:30.131-04:00Intro to ClojureScript - Getting Started <p>I’ve been curious about ClojureScript for awhile now but for whatever reason I hadn’t taken the plunge. A few weeks back I attended a at talk at a TriClojure meetup titled <a href="http://www.meetup.com/TriClojure/events/71707312/">‘A ClojureScript Experience Report’</a>. Jason broke down his experience creating an app using ClojureScript ins such a way that it motivated me to start learning ClojureScript.</p>
<p>
This is the first in a series of posts that I will write that chronicles my journey down the ClojureScript path. I will be building out a basic app to lookup hockey stats by the last name of a player. Keep in mind I’m learning ClojureScript as I write these posts. If you have experience with ClojureScript and you see me making errors or not doing things in a idiomatic way please feel free to point out the errors of my ways.
</p>
<p><em>
Since ClojureScript is a bit of a mouthful I will use the abbreviation cljs for the remainder of this post.
</em></p>
<h3>The Goal</h3>
<p>
By the end of this post you will have an environment that is ready to create cljs apps. You will also have the beginnings of our hockey stats lookup app.
</p>
<h3>The Setup</h3>
<p>
Before you can start writing you app you’ll need to get your environment setup. The first step is to get <a href="http://leiningen.org/#install">Leiningen installed</a>. Follow the directions on the page and you’ll be ready to create your first project.
</p>
<p>
Creating a cljs project is done the same way you create a Clojure project:
</p>
<code>lein new cljs-intro</code>
<p>
This will create Clojure project with the following layout:
</p>
<pre>
README
project.clj
src
cljs_intro
core.clj
test
cljs_intro
test
core.clj
</pre>
<p>
In order to keep our Clojure and cljs code seperate we need to change the directory layout of our project a little bit. We need to add two directories, move the existing Clojure code and create the directory for housing our HTML and cljs compiler output. If you are in the project’s home directory run the following commands:
</p>
<ol>
<li>mkdir src/clj src/cljs</li>
<li>mv src/cljs_intro src/clj</li>
<li>mkdir -p resources/public</li>
</ol>
<p>
Now that we’ve created the directories and moved the code we need to update the project.clj file to reflect these changes and add cljs specific settings.
</p>
<p>
The project.clj file generated by the lein command looks like this:
</p>
<script src="https://gist.github.com/3621574.js?file=gistfile1.clj"></script>
<p>
First we need to add the plugin lein-cljsbuild to the project. lein-cljsbuild is the plugin that makes it possible to compile cljs apps. Adding lein-cljsbuild to our is as simple as adding the :plugin line to the project file. Here’s what the updated project.clj file looks like after adding the line.
</p>
<script src="https://gist.github.com/3621578.js?file=gistfile1.clj"></script>
<p>
Since we moved the source code location we need to update the project.clj file to reflect those changes. By adding the following line the file we inform lein where the Clojure code lives.
</p>
<pre>:source-path "src/clj"</pre>
<p>
Now its time to set up the cljs related settings. Here’s what we need to add to the project file:
</p>
<script src="https://gist.github.com/3621584.js?file=gistfile1.clj"></script>
<p>
Here we are we are telling the cljs compiler to output the results of the compilation into the file resources/public/hockey.js. Since we have an optimization setting of :whitespace all compiled javascript will be placed into a single file. There are four levels of optimization levels available: :none, :whitespace, :simple and :advanced. We will stick with the :whitespace option. The :whitespace setting removes comments and any unnecessary whitespace in the source code but leaves the remaining code alone. The book <a href="http://shop.oreilly.com/product/0636920025139.do">ClojureScript: Up and Running</a> has a nice discussion about optimization. The :pretty-print option gives us output that is a little easier to read. Here’s what the updated project.clj file looks like:
</p>
<script src="https://gist.github.com/3621608.js?file=gistfile1.clj"></script>
<p>
To ensure we have everything correct in the project.clj file go ahead and start the Clojure repl by running this command:
</p>
<pre>lein repl</pre>
<p>If everything is OK you should see something like this:</p>
<pre>REPL started; server listening on localhost port 42236
user=></pre>
<h3>Time to Write Code</h3>
<p>
Now that we have our project setup for cljs its time to start writing some code. For this post we will create the search form which will have a text field and a button. The ‘back end’ of the form will be written using cljs and will be responsible for responding to the search button click. When the button is clicked the cljs code will grab the value of the text field and displaying it in an alert box.
</P>
<h2>resources/public/index.html</h2>
<p>Before we write the cljs code lets create the HTML page. Its a simple form that has a text box and a button. Here’s what the HTML looks like:</p>
<script src="https://gist.github.com/3621553.js?file=index.html"></script>
<p>The only real line of note here is the script tag. It references the file that we will generate when we compile our cljs code.
</p>
<h2>src/cljs/search.cljs</h2>
<p>
The src/cljs/search.cljs file is where we will put the code to handle the user’s input. Once you have created the file open it in your favorite editor and add the following lines:
</p>
<script src="https://gist.github.com/3621616.js?file=gistfile1.clj"></script>
<p>
The above code sets the namespace for the code and brings in the domina and clojure.browser.event libraries. We need the domina library so we can interact with the DOM. The clojure.browser.event library is used to add the event handler for the button’s click event.</p>
<p>
Before we write the search button's click event handler lets get a reference to the search button itself.
</p>
<code> (def search-button (d/by-id "search-btn")) </code>
<p>
We get a reference to the button by making use of domina's by-id function and store the reference to the DOM object in search-button. Now its time to create the event handler.
</p>
<script src="https://gist.github.com/3621666.js?file=gistfile1.clj"></script>
<p>The clojure.browser.event/listen function creates the relationship between the search button’s click event and our handler function. We create a function to popup an alert box containing the contents of the text field.
</p>
<h3>Compile Time</h3>
<p>
To compile the cljs code we have two options: lein cljsbuild once or lein cljsbuild auto. The once option compiles our code like a ‘normal’ compiler. It runs, compiles the source and puts the output to the file that we indicated in the :output-to setting. If we compile with auto every time a cljs file is saved the compiler will run. For now we’ll stick with the once option. To compile our code go to anywhere in the project’s directory structure and enter:
</p>
<code>lein cljsbuild once</code>
<p>
Now open up the the HTML file we created in the browser and you should see a beautifully designed form that looks something like this:
</p>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-J3zHRz4lbFw/UEYTmegPSfI/AAAAAAAAAs8/Yaj0-qXisrI/s1600/form.png" imageanchor="1" style="margin-left:1em; margin-right:1em"><img border="0" height="52" width="235" src="http://1.bp.blogspot.com/-J3zHRz4lbFw/UEYTmegPSfI/AAAAAAAAAs8/Yaj0-qXisrI/s320/form.png" /></a></div>
<p>
Enter the string ‘ClojureScript!’ in the text field and click the ‘Get Stats!’ button. You should see something like this:
</p>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-VXXaxOgsPMc/UEYTyxDS4VI/AAAAAAAAAtI/1kXFNtUeQ-U/s1600/alert.png" imageanchor="1" style="margin-left:1em; margin-right:1em"><img border="0" height="167" width="320" src="http://2.bp.blogspot.com/-VXXaxOgsPMc/UEYTyxDS4VI/AAAAAAAAAtI/1kXFNtUeQ-U/s320/alert.png" /></a></div>
<p>
We now have the beginnings of our stats lookup app. The user interaction is in place but we don’t really have a lot of the ‘under the covers’ code in place. In my next post we will add code to the hockey database and run an actual query!
</p>
<h3>Summary</h3>
<p>
In this post we went over how to get your environment set up for ClojureScript. We also went over the necessary changes to the project directory structure and project.clj file. We also created a simple cljs file to handle the user’s input in our search form.
</p>
<h3>Whats Next?</h3>
<p>
In <em>Intro to ClojureScript - Part 2 - Getting the Stats</em> we will create the necessary back end to retrieve data from the database and display it on the page.
</p>
<p>
In <em>Intro to ClojureScript - Part 3 - Incorporating ClojureScriptOne</em> - We will add support for messaging. We will add a new feature with messaging.
</p>
<h3>Resources</h3>
<p>
<a href="http://clojurescriptone.com/">ClojureScriptOne.com</a> - A sample project to help people get familiar with ClojureScript.
</p>
<p>
<a href="http://jasonrudolph.com/blog/2012/08/17/clojurescript-experience-report-resources/">ClojureScript Experience Report - Resources</a> - Bits of information about Jason’s ClojureScript based application and ClojureScript.
</p>
<p>
<a href="http://shop.oreilly.com/product/0636920025139.do">ClojureScript: Up and Running</a> - An early release book that discusses ClojureScript. I’ve read the released chapters and found it to be a good resource. Chapters 2 and 3 go over ClojureScript’s compilation, project structure, and other informative tidbits for both ClojureScript and Clojure. I’ve recommended the book to a colleague who is learning Clojure because of the way the authors describe data structures, immutability and sequences.
</p>
<p>
<a href="https://github.com/rippinrobr/cljs-intro">cljs-intro</a> - My github repo that stores the code for this blog series.
</p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com8tag:blogger.com,1999:blog-724993853864308794.post-28611909936641205462012-04-11T16:05:00.001-04:002012-04-11T16:05:40.116-04:00Creating a TFS Work Item from ClojureCLR<p>At my day job we use Team Foundation Server 2008 (TFS) for our automated builds, iteration management, and source control. TFS may not be the most ideal way to manage these processes but in our MS environment it has helped us communicate with our non-technical team members and customers.  So we’ve been looking into ways to add bugs automatically when automated tests fail (see <a href="http://progadventure.blogspot.com/2012/04/creating-tfs-work-item-from-ironruby.html">Creating A TFS WorkItem From IronRuby</a>) or errors are reported from our applications via a message queue.</p> <p>This morning I had a little time to do some ‘research’ on how to programmatically create a new bug work item.  My goal was to write code that would create a new work item bug with an image attached to it.  Before I get into the code let me take a second to introduce you to a TFS Bug Work Item.</p> <h5>A TFS Bug Work Item</h5> <p>In TFS Work Items are a way to track work that needs to be done. There are a total of five different type of Work Items available but we typically only use three: Task, Scenario, and Bug. Each Work Item type UI has different fields. Here is what a Bug Work Item looks like in our environment.</p> <p><a href="http://lh5.ggpht.com/-ogmwpKWY1Ys/T4WUgNcvf7I/AAAAAAAAArU/DueDk_NHwyI/s1600-h/empty-bug-wi-tfs%25255B3%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="empty-bug-wi-tfs" border="0" alt="empty-bug-wi-tfs" src="http://lh5.ggpht.com/-1PMQCNm-k9U/T4WUgrv5e_I/AAAAAAAAArc/83i397zc0rw/empty-bug-wi-tfs_thumb.png?imgmax=800" width="568" height="366" /></a> </p> <p>In this example we will create a new bug and enter text into the highlighted fields plus attach an image file. In order to create the bug I need to do a little setup.</p> <h4>The Setup</h4> <p>There are three assemblies needed to create a TFS Bug Work Item. The are: Microsoft.TeamFoundation.dll, Microsoft.TeamFoundation.Client.dll, and Microsoft.TeamFoundation.WorkItemTracking.Client. All three of these DLLs can be found in the C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies directory. I copied the three Dlls into the project’s libs directory. </p>
<h4>The Code</h4>
<script src="https://gist.github.com/2304442.js?file=gistfile1.clj"></script>
<p>The first step is to load the TFS assemblies from libs directory using the assembly-load-from function. I follow that call up with the import function pulling in the specific classes I need.  Once the assemblies and classes have been loaded I can start working on creating a bug work item. First, I created the -main function. The first few lines of –main are responsible for creating the necessary objects for WorkItem creation.</p>
<script src="https://gist.github.com/2361960.js?file=gistfile1.clj"></script>
<p>The first line gets the server object by making a call to the static method TeamFoundationServerFactory.GetServer passing in the name of the server I am working with.  The returned server object is used to create a WorkItemStore object.  A WorkItemStore encapsulates the data store that contains all work items on a particular server.  The next line gets the TFS project that I want to work with by using the filter function.  The last setup line uses the project object to retrieve the WorkItemType object that represents a bug.  Now I 'm are ready to create a new bug WorkItem.</p>
<script src="https://gist.github.com/2362040.js?file=gistfile1.clj"></script>
<h5>Creating the Bug WorkItem</h5>
<script src="https://gist.github.com/2306424.js?file=gistfile1.clj"></script> <p>Since I will be creating bugs that have attachments and bugs that do not I created a function that creates and returns a WorkItem object. As I stated previously, the work items created this way will have the bare minimum values: title, area, iteration and description. The code is straight forward, just create the WorkItem object by passing in the WorkItemType object. Next I set the properties of for the four parameters. When everything has been set I return the WorkItem object.</p> <h5>Adding the Attachment</h5> <script src="https://gist.github.com/2354232.js?file=gistfile1.clj"></script> <p>Now that I have my WorkItem object its time to add my image file as an attachment. I wrote another small and simple function to create the Attachment object. It takes three parameters, the WorkItem, the path to the file to attach and a description of the file.  First I use the path and desc parameters to create the Attachment object.  After the Attachment is created I can added it to the WorkItem object’s Attachments collection by calling the Add method passing the Attachment object as its only parameter.  </p> <p>At this point I have a WorkItem object with an Attachment in memory.  At this moment the WorkItem is not visible through the TFS toolset.  I still have one last line of code to write to ensure that the WorkItem is saved. Surprisingly enough all I need to do is call the WorkItem’s save method.</p> <code>(.Save work-item)</code> <p>Now, If I look at the Pending Bugs report in TFS I will see my newly created bug in the list.</p> <p><a href="http://lh4.ggpht.com/-6Py3ZjZtJlw/T4WUhPjODDI/AAAAAAAAArk/BW7tsGtErlg/s1600-h/pending-bugs-clj-ir%25255B3%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="pending-bugs-clj-ir" border="0" alt="pending-bugs-clj-ir" src="http://lh4.ggpht.com/-KrRkmaPIRao/T4WUhiMo7tI/AAAAAAAAArs/j_40QtfPwQc/pending-bugs-clj-ir_thumb%25255B1%25255D.png?imgmax=800" width="615" height="167" /></a></p> <h4>Summary</h4> <p>This post is another illustration of how easy it is to use .NET Assemblies from ClojureCLR.  In this case I walked you through the simple process of creating a Bug WorkItem in TFS.  Having the ability to programmatically create and report bugs will help us close the feedback loop in our applications.  </p> <h4>In The Future</h4> <p>In my ‘free time’ I will be checking into TFS’s build and source control APIs to see what we may be able to do improve our processes.  If and when I find anything of interest I will be sure to do a post or two about my findings.</p> <h4>Resources</h4> <p>TFS API: <a title="http://msdn.microsoft.com/en-us/library/bb130146(v=vs.90).aspx" href="http://msdn.microsoft.com/en-us/library/bb130146(v=vs.90).aspx">http://msdn.microsoft.com/en-us/library/bb130146(v=vs.90).aspx</a></p> <p>My Source (this blog’s code is the 5-create_tfs_work_item):<a href="https://github.com/rippinrobr/clojure-clr-intro/zipball/master">https://github.com/rippinrobr/clojure-clr-intro/zipball/master</a></p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com1tag:blogger.com,1999:blog-724993853864308794.post-67965406523433406782012-02-15T07:48:00.000-05:002012-02-15T07:48:33.465-05:00My First Attempt at Porting Libraries to ClojureCLR<p>A while back David Miller wrote a couple of blog posts (<a href="http://clojureclr.blogspot.com/2012/01/porting-libs-to-clojureclr-example.html">Porting libs to ClojureCLR: an example</a> and <a href="http://clojureclr.blogspot.com/2012/01/porting-effort-for-clojure-contrib-libs.html">Porting effort for Clojure contrib libs</a>) about porting Clojure contrib libs to ClojureCLR.  In the two posts he made it look like porting most of the libs wouldn’t be too terribly hard.  After a long period of telling myself, “tomorrow I will pick a lib to port and get started”, I actually did do just that a few days ago. I am about to start a project that needs to process command line arguments and it just so happens that David flagged the tools.cli lib as a trivial port.  So I decided to start my porting career with tools.cli.</p> <h4>The Setup</h4> <p>I created a directory off of my usual code directory called clj-ports just to help me differentiate that code from everything else I’m working on. Next I cloned the existing github project (<a href="https://github.com/clojure/tools.cli">https://github.com/clojure/tools.cli</a>).  After cloning the project I attempted to compile the project with clojure.compile. For tools.cli the compilation went off without a hitch. Now that I have a clojure.tools.cli.clj.DLL  I’m ready to start testing.</p> <h4>The Testing Process</h4> <p>Now that I have my DLL I’m ready to start up the REPL and run the tests. The first step to running the is to load the clojure.test and the clojure.tools.cli-test libraries. After loading the libraries I can run the tests.  When I ran the tests and very quickly I hit my first error.</p> <script src="https://gist.github.com/1827293.js?file=gistfile1.txt"></script> <p>The error was a System.MissingMethodException on a call to a split method. I scoured the cli.clj file for a call to split but couldn’t find one. After taking a closer look I realized it was in the TESTING code. I opened up cli-test.clj, changed split to Split. When I re-ran the tests I hit another error. This time the test failed because it expected a Char[] and was passed a String.</p> <script src="https://gist.github.com/1827419.js?file=gistfile1.txt"></script> <p>The problem is in the testing code and is one of the parameters to the Split method call.  Since it was looking for a Char[] and I’m passing in a one character string I thought I’d try to use (first “-“) to get around the issue.  So I updated the test code and the change comment and re-ran the tests.  The error message I received this time was in the cli.clj file and not the test file.  Now that I have resolved the issues in the test file I can add my end of line comments following David’s instructions in his post <a href="http://clojureclr.blogspot.com/2012/01/porting-effort-for-clojure-contrib-libs.html">Porting effort for Clojure contrib libs</a>. </p> <p>The new error is:</p> <p><code>actual: System.MissingMethodException: Cannot find member startsWith matching args at .CallSite.Target (:0)</code> </p> <p>I opened up the cli.clj file and searched for startsWith calls. I found four places where the java startsWith method was used.  I changed all four to use the .NET equivalent StartsWith (and of course I commented the lines accordingly).  Remember, after you make a change to the source file you will need to re-compile (at least that is the way I did it) which means you’ll need to exit out of your REPL session first.  After recompiling and getting my REPL session back in order I re-ran the tests.  This time they all PASSED! </p> <p>Now that I have all greens on my test it is time to create a new project on github called cljclr.tools.cli. After creating the project on github and setting up my machine I copied all files from the clojure.tools.cli project into the new cljclr.tools.cli project.  Next, I changed the namespaces in all code files to reflect the new project name and re-ran the tests.  All green! I checked the ported code into github and called the port complete. The ported project lives at <a href="https://github.com/rippinrobr/cljclr.tools.cli">cljclr.tools.cli</a>.</p> <h4>Sample App</h4> <script src="https://gist.github.com/1827539.js?file=gistfile1.clj"></script> <p>Since this was my first time porting a library I wanted to make sure it worked I used the ‘application’ above to check.  The first part of the main function is where the call to the cli function is which is where the available command line options are configured.
The parameters are the args passed to the main function followed by a vectors that illustrate the allowed command-line options.  The returned vector has three members, a map with the parameters and their values, a vector with any additional parameters passed on the command line, and the string that contains the ‘banner’ which is used to show the available parameters.  After the call the cond function  is used to figure out what to do.  Each option has a simple println if one of the available parameters is passed in.  If no parameters are passed the usage banner is printed.  After compiling and running the app I was convinced that the port worked!  </p> <h4>Since That was Easy…</h4> <p>I thought I’d try my hand at another lib.  This time I chose to port tools.macro.  I went through the same configuration process, cloning, compiling and starting up the REPL so I can run the tests.  This time I hit NO errors when I ran the tests. That is the easiest type of port!  </p>
<p>Like a gambling addict I pressed my luck and chose one more library to port.  Since the algo.monads library was flagged as trivial I grabbed it.  When I ran the tests I was hoping for the same results as the tools.macro port but this time I hit an error.  The first error was another simple swap a java class for a .NET class: java.lang.IllegalArgumentException for System.ArgumentException and re-ran the tests.  The second error was in an extend-protocol call  using java.lang.String.  I simply swapped the java.lang.String for a System.String call.  When I ran the tests again all tests passed.  Another port complete!  The project can be found at <a href="https://github.com/rippinrobr/cljclr.algo.monads">cljclr.algo.monads</a> on github.</p> <h4>Summary</h4> <p>Porting libraries to ClojureCLR wasn’t nearly as time consuming or complicated as I thought it would be.  In fact I have probably spent more time on writing this blog post than I did doing the ports.  Of course I picked libraries flagged as trivial ports which is probably why it didn’t take long to finish.  What porting these libs over did teach me was the process: Clone, Compile, Test, fix, retest.  For my next porting attempt, the data.csv library I will add a step to search the cloned files for easy fixes like startsWith to StartsWith and other mappings that David laid out in his posts.  Please join the porting party and port one today!  I have not tested the ports against 1.2, all of my ports were done with 1.3.  I will continue to test only with 1.3 unless others feel that we need to test against 1.2.  If you use any of the libraries I’ve ported and run into a bug please do not hesitate to let me know.</p> <h4>Resources</h4> <p><strong>Blog Posts: </strong><a href="http://clojureclr.blogspot.com/2012/01/porting-libs-to-clojureclr-example.html">Porting libs to ClojureCLR: an example</a> and <a href="http://clojureclr.blogspot.com/2012/01/porting-effort-for-clojure-contrib-libs.html">Porting effort for Clojure contrib libs</a></p> <p><strong>Ported Projects: </strong><a href="https://github.com/rippinrobr/cljclr.tools.cli">cljclr.tools.cli</a> and <a href="https://github.com/rippinrobr/cljclr.algo.monads">cljclr.algo.monads</a></p> <p><strong>Projects that work ‘As Is’</strong>: <a href="https://github.com/clojure/tools.macro">clojure.tools.macro</a></p> <p>In order to help others see which libraries have been ported and/or tested for ClojureCLR I have created a page that lays out which libraries have been ported and the location of the github project. If you have ports you’d like listed on the page please let me know. <a href="http://www.myclojureadventure.com/p/clojureclr-ports-page-purpose-of-this.html">The ClojureCLR Ports Status Page is here</a>.</p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com2tag:blogger.com,1999:blog-724993853864308794.post-41770644616312737642012-01-11T11:05:00.002-05:002012-01-11T11:05:32.595-05:00Intro to Clojure-clr: Creating a UI with Windows.Forms<p>In this post I will walk you through the creation of a Windows.Forms based user interface using ClojureCLR. </p>
<h3>The Setup</h3>
<p>To follow along with the blog you will need to have ClojureCLR installed and have access to a MySQL and SQL Server instances. If you haven’t installed it yet follow the directions in my <a href="http://www.myclojureadventure.com/2011/10/getting-started-with-clojure-clr.html" target="_blank"><em>Getting Started with Clojure-clr</em></a><em> </em>post. If you need to install one or both of the bases you can download and install them from here: <a href="http://www.microsoft.com/sqlserver/en/us/editions/express.aspx">SQL Server Express</a> and <a href="http://dev.mysql.com/downloads/installer/">MySQL</a>.  Once you have downloaded and installing the databases the last step you’ll need to take is to grab the <a href="https://github.com/rippinrobr/clojure-clr-intro/blob/master/4-windows-forms/sql/mysql_schema.sql" target="_blank">mysql_schema.sql</a> file and run it in your MySQL database. If you are not familiar with MySQL you can run the file from the command line by running this:</p>
<pre>mysql –user=<em>username</em> –password=<em>password</em> < mysql_schema.sql</pre>
<p>This will create a database and three tables: Master, Schools and PlayerSchools. The three tables will be migrated into a new SQL Server database. These tables are from a database created by the group <a href="http://www.baseball-databank.org/" target="_blank">baseball-databank.org</a> who have collected baseball statistics from the beginning of professional baseball to the present. I find using this data more exciting then the usual demo data.  Anyway, once you have the loaded the schema into your MySQL instance you are ready to start building the user interface.</p>
<h3>UI.clj</h3>
<p>Surprisingly enough the ui.clj file is where the user interface code lives. Here’s what it looks like when it is first started.</p>
<p><a href="http://lh4.ggpht.com/-40pIWFPXgsk/Twy6d7Doz8I/AAAAAAAAAo0/kUXSUIwjbmc/s1600-h/image3.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; margin-left: auto; border-left-width: 0px; margin-right: auto; padding-top: 0px" title="image" border="0" alt="image" align="left" src="http://lh3.ggpht.com/-Y9Zja33J_bY/Twy6eS950KI/AAAAAAAAAo8/u9G3l9565_I/image_thumb1.png?imgmax=800" width="317" height="363" /></a></p>
<p>I won’t bore you with a line by line analysis of the source but I will show you highlights of the UI creation process, walking you through the steps below. Creating Windows.Forms apps follow the same process without regard to the language that they are being creating in. So there aren't any magic steps here. However, creating the UI in Clojure was a lot of fun.</p>
<h4>Step 1. Including the necessary assembly and classes</h4>
<script src="https://gist.github.com/1584097.js?file=gistfile1.clj"></script>
<p>The first step in creating the UI is to bring in the System.Windows.Forms assembly which I did by calling LoadWithParitalName. Next, I used :require to bring in the classes I needed create the UI with. Since I want to be able to run this app outside of the REPL, I call :gen-class to generate ui.exe.</p>
<h4>Step 2. Instantiating the UI Objects</h4>
<script src="https://gist.github.com/1584520.js?file=gistfile1.clj"></script>
<p>After I have loaded and required everything I need I can start creating objects. As you can see the creating objects is pretty straight forward, all I do is new up all the objects. In the next step is where the configuration of the objects and the event handlers are added.</p>
<h4>Step 3. Setting Properties</h4>
<script src="https://gist.github.com/1585019.js?file=gistfile1.clj"></script>
<p>The snippet above contains a sampling of the property setting code for the title label, the group box, the CheckedListBox and the 'Load Table' button. If you’d like to see the entire file you can do so <a href="https://github.com/rippinrobr/clojure-clr-intro/blob/master/4-windows-forms/ui.clj" target="_blank">here</a>. You may notice all the calls to methods with names like set_Text and wonder where the methods come from since you do not see them when you look at the Label class in the Object Browser. In ClojureCLR we access the .NET objects at the CLR 'layer' which is different than the we access them through C#. The set_Text method is the CLR representation of the C# code Label.Text = "Some Text". If you’d like to see what other methods are available you can use the ildasm tool, which is a part of the .NET SDK. Here is an example of some of the set_ methods for the Label class:</p>
<p><a href="http://lh6.ggpht.com/-XWgFY2_ZG5I/Twy6ehazttI/AAAAAAAAApE/YzOKpdscDaA/s1600-h/image11.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; margin-left: auto; border-left-width: 0px; margin-right: auto; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-sYO8BnN8qxc/Twy6fGRT22I/AAAAAAAAApM/nYzzCZdf_IE/image_thumb7.png?imgmax=800" width="498" height="138" /></a></p>
<h4>Step 4. Adding the UI Objects to the Form and displaying the it</h4>
<script src="https://gist.github.com/1585279.js?file=gistfile1.clj"></script>
<p>Now that I have all the objects customized I would like to show them to the user. In order to do that I need to add them to the Form object. The Form class has a Controls collection where all of its child objects are stored. I added the objects to the collection by calling the Controls.Add method for each object. When all of the objects have been added I call the ShowDialog method to display the UI.</p>
<h4>Step 5. Adding Click Event Handlers to the Buttons.</h4>
<p>A snazzy UI is great but if it doesn’t do anything what good is it? Handling the Button.Click events for both of the buttons is where all the action happens. In order to wire up the Button.Click event handlers I needed to use the gen-delegate macro. The macro’s signature looks like this: </p>
<pre>gen-delegate<br />([type argVec & body])</pre>
<h2>The Load Table Click Event Handler</h2>
<p> Here's what the 'Load Table' button's click handler looks like:</p>
<p><script src="https://gist.github.com/1585363.js?file=gistfile1.clj"></script></p>
<p>In my call to gen-delegate the parameters begin with passing the type System.EventHandler since we want a EventHandler delegate created. The next argument is the a vector which in this case contains the usual Click event parameters: sender and args. The last parameter is where the code of the event handler reside. In this case when the 'Load Table' button is clicked all of the table names from my bdb_post_2010 database are returned and are added to the CheckedListBox. After clicking the 'Load Table' button the app looks like this for me. <em>Your table listing will look different.  The SQL File I provide only has three tables.</em></p>
<p><a href="http://lh6.ggpht.com/-MWiT_C6YHOw/Twy6fd-QPeI/AAAAAAAAApU/YAmgIpu9pAY/s1600-h/image17.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; margin-left: auto; border-left-width: 0px; margin-right: auto; padding-top: 0px" title="image" border="0" alt="image" align="left" src="http://lh3.ggpht.com/-DyBPvZo5IYc/Twy6f0PP9bI/AAAAAAAAApc/uRejwoVUviY/image_thumb11.png?imgmax=800" width="369" height="443" /></a></p>
<p>The tables are retrieved by creating a MySQL connection which is then passed to a function that retrieves the list of tables in the specified database. </p>
<p>The returned lazy sequence I passed through doseq so the tables can be added to the table list.  The table names are added by making a call to CheckedListBox.Items.Add method.  Once all of the table names have been added I close the database connection.</p>
<h2>The Migrate Tables Event Handler</h2>
<p>The ‘Migrate Tables…’ button’s event handler does a similar task as the ‘Load Table’ handler does, except it interacts with both MySQL and SQL Server databases.  </p>
<p>First it creates a connection to the MySQL database and uses the value from the 'New DB Name' TextBox to create the new database on the SQL Server instance. Next, doseq is called to process the values in the CheckedListBox.CheckedItems collection. Each of the selected table names is passed to mysql/get-columns to retrieve a sequence of maps that contains information about each column in the table. The returned sequence is then passed with the table name and SQL Server database object to sql/create-table which to creates the table in the SQL Server database. After all of the selected tables have been migrated to SQL Server the MySQL connection is closed and a MessageBox is displayed to inform the use that the migration is complete.</p>
<h2>db/core.clj and db/sqlserver.clj</h2>
<p>Since this is a blog post about creating a UI I will not be delving into the minute details of how the database related code works. However, I do think it is worth giving you a quick overview of the code. The DB code in db/mysql.clj, db/core.clj and db/sqlserver.clj are based on the code I wrote in <a href="http://www.myclojureadventure.com/2011/11/intro-to-clojure-clr-connecting-to-sql.html" target="_blank"><em>Intro to Clojure-clr: Connecting to SQL Server and MySQL</em></a>.</p>
<p>The db/core.clj file contains one function named run-sql which does what you might expect, it executes a query. In my previous database connectivity post I used (.Read reader) to loop through the results of a query but here I've switched to using the System.Data.DataTable class. Using the DataTable class allows me to close the reader right away and retrieve the column names for the query very easily. I used the column names with the zipmap function to map the column name to its values for each of the returned rows. This allows me to return a sequence of maps that represented the results of the query.</p>
<p>The db/mysql.clj file has three methods: get-connection, get-tables and get-columns. Each of these functions does what their names imply. get-tables uses 'show tables;' and get-columns uses 'describe [table name];'.</p>
<p>On the migration side of the app, the db/sqlserver.clj file makes use of the Microsoft.SqlServer.Smo assembly to create the new database, tables and columns. If you want more information on the visit the <a href="http://msdn.microsoft.com/en-us/library/ms212724.aspx" target="_blank">Microsoft.SqlServer.Management.Smo MSDN Page</a>.</p>
<h4>Running the App</h4>
<p>Starting the application from the REPL is straight forward.  Start the REPL in the same directory that as the ui.clj file. When the REPL is ready enter:</p>
<p><a href="http://lh6.ggpht.com/-Ql5dGwd-ImM/Twy6g-CKplI/AAAAAAAAAp0/b7-MO-liZ6w/s1600-h/image24.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; margin-left: auto; border-left-width: 0px; margin-right: auto; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-cKlr3ysPli4/Twy6hbZ4qMI/AAAAAAAAAp8/mFnWWOqV2XQ/image_thumb14.png?imgmax=800" width="230" height="83" /></a></p>
<p>You should then see the UI appear with an empty CheckListBox.  Once the app is up and running click on the ‘Load Table’ button.  If are used the SQL file I mentioned above you should see three tables:  master, schools, and schoolsplayers.</p>
</p>Go ahead and click all three tables.  You will need to click on the table once to highlight it and then either click on the check box or press the space bar to put a check in the box.</p>
<p><a href="http://lh3.ggpht.com/-BA4xv0YRIeQ/Twy6gNDSPrI/AAAAAAAAApg/ZGpNfl8Lkd0/s1600-h/image21.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; margin-left: auto; border-left-width: 0px; margin-right: auto; padding-top: 0px" title="image" border="0" title="image" border="0" alt="image" align="left" src="http://lh4.ggpht.com/-QoWxamKPGTY/Twy6gRB-ayI/AAAAAAAAApo/TLrLsdX_z7I/image_thumb13.png?imgmax=800" width="339" height="397" /></a></p>
<p>Next, enter the new database name.  My database will be named clr-intro-4. Finally, I kick off the migration click on the ‘Migrate Tables…’ button and within a few seconds you will have a new database with the three tables added. When the migration has completed you will see something similar to the image below.</p>
<p><a href="http://lh4.ggpht.com/-bwYEJuAV6VE/Twy6hqDd6xI/AAAAAAAAAqE/2NiATsUXIiI/s1600-h/image28.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; margin-left: auto; border-left-width: 0px; margin-right: auto; padding-top: 0px" title="image" border="0" alt="image" align="left" src="http://lh4.ggpht.com/--jcTchBqPDI/Twy6iPvvFPI/AAAAAAAAAqM/fJbDGO0nNns/image_thumb16.png?imgmax=800" width="314" height="433" /></a></p>
<h3>Summary</h3>
<p>Building a UI with System.Windows.Forms in ClojureCLR is straight forward. You only need to bring in the System.Windows.Forms assembly and follow the typical process of creating a Forms based application. </p>
<p>Adding event handlers is as easy as making use of the gen-delegate macro passing it the ‘guts’ of what you want to do when a particular event is fired.</p>
<h3>About the Data</h3>
<p>The data I’m using for this blog post comes from the <a href="http://www.baseball-databank.org/">Baseball Databank</a> project.  The project has gathered all the baseball status from previous season and offers the data in many different formats.</p><br /><br /><br /><br /><br /><br /><br />
<h3>Resources</h3>
<p><strong>Previous Posts:</strong> <a href="http://www.myclojureadventure.com/2011/10/getting-started-with-clojure-clr.html" target="_blank"><em>Getting Started with Clojure-clr</em></a>, <a href="http://www.myclojureadventure.com/2011/11/intro-to-clojure-clr-connecting-to-sql.html" target="_blank"><em>Intro to Clojure-clr: Connecting to SQL Server and MySQL</em></a></p>
<p><strong>Database Downloads:</strong> <a href="http://www.microsoft.com/sqlserver/en/us/editions/express.aspx">SQL Server Express</a>, <a href="http://dev.mysql.com/downloads/installer/">MySQL</a></p>
<p><strong>Data:</strong> <a href="https://github.com/rippinrobr/clojure-clr-intro/blob/master/4-windows-forms/sql/mysql_schema.sql" target="_blank">mysql_schema.sql</a></p>
<p><strong>My Source</strong>: <a href="https://github.com/rippinrobr/clojure-clr-intro/zipball/master">https://github.com/rippinrobr/clojure-clr-intro/zipball/master</a></p>
<p><strong>More Examples and Information:</strong> <a href="https://github.com/clojure/clojure-clr/blob/master/Clojure/Clojure.Source/clojure/samples/celsius.clj" target="_blank">Another Windows Forms Example</a>, <a href="http://msdn.microsoft.com/en-us/library/ms212724.aspx" target="_blank">Microsoft.SqlServer.Management.Smo MSDN Page</a>, and <a href="http://msdn.microsoft.com/en-us/library/f7dy01k1(v=VS.100).aspx" target="_blank">Ildasm.exe page on MSDN</a></p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com4tag:blogger.com,1999:blog-724993853864308794.post-23208243892512888042011-12-22T07:00:00.000-05:002011-12-22T08:06:34.917-05:00Intro to Clojure-clr: Calling Clojure from C#<p>In today’s post I am going to show you how to call Clojure functions from a C# project.  </p> <h4>The Setup</h4> <p>The requirements for this post is to have ClojureCLR installed and to have access to a C# compiler.  If you haven’t installed Clojure-clr 1.3 you can do so by following the steps in my previous post <a href="http://www.myclojureadventure.com/2011/10/getting-started-with-clojure-clr.html" target="_blank">Getting Started with Clojure-clr</a>.  If you do not have a C# compiler already you can grab <a href="http://www.microsoft.com/visualstudio/en-us/products/2010-editions/visual-csharp-express" target="_blank">Visual C# 2010 Express</a> or <a href="http://monodevelop.com/Download" target="_blank">MonoDevelop</a>.  I used ClojureCLR 1.3 and ClojureCLR 1.4-master-snapshot for this post.  I will explain the ClojureCLR 1.4 snapshot in moment.  For the C# portion of the post I used Visual Studio 2010 Ultimate.  I haven’t tried using MonoDevelop but I’m sure it would work fine for the C# portion of this post.</p> <h4>The Clojure Code</h4> <p>I have written a couple of functions in Clojure that I would like to re-use in a C# project.  The functions are ba which will calculate a batting average and standings which will tell me where a particular team finished in Major League Baseball’s National League West division.  </p> <script src="https://gist.github.com/1486013.js?file=gistfile1.clj"></script> <p>There isn’t much to either one of the Clojure functions but there are a few things I’d like to point out in the ns statement, mainly the :gen-class call and the :methods options passed to it. The :gen-class call is what forces the generation of the executable, without it only DLLs would be created when the code is compiled.  The :methods options us a way to indicate which functions should be exposed as methods in the generated .NET class. :methods expects a vector of method signatures.  </p> <script src="https://gist.github.com/1486061.js?file=gistfile1.clj"></script> <p>The method signature is prefaced with meta data to indicate that I want the methods to be static in the generated class. The exposed method's signature is described by a vector that follows the pattern below:</p> <pre>[method name [parameters] return value]</pre>
<p>As you can see the ba method takes two int parameters and returns a double and the standings method takes a string and returns a string. </p>
<br /><script src="https://gist.github.com/1486148.js?file=gistfile1.clj"></script>
<p>By default methods listed in :methods will be mapped to a Clojure function with the same name prefixed by a –. You can change the function’s prefix by passing the :prefix option to :gen-class with your desired prefix. As an example if I wanted to prefix the functions with csharp- I would pass :prefix “csharp-“. Then ba would map to a function of csharp-ba and the standings function would be csharp-standings.  In this example, I’m using the – functions to echo the parameters the function was called with and then I call the real functions.  </p>
<p>You may have noticed that the –ba and –standings functions have a parameter named dummy.  The dummy parameter is there as a work-around.  When I first started working on calling Clojure from C# I noticed that a function that did math was returning the incorrect value when called from C#.  If I ran the code with the same parameters from the REPL or after I compiled the code and ran it from the command line it worked fine.  I was only having the problem when I called the functions from C#.  To figure out what was going on I opened the clj file in Visual Studio and set a debug point in the –ba function. Here’s an example of stopping at a breakpoint in Visual Studio.  If you have VsClojure installed you can debug clojure code in the same way you can C#.  <em>You can see how to get clojure support in Visual Studio in my blog post <a href="http://www.myclojureadventure.com/2011/03/getting-vsclojure-up-and-running-in.html" target="_blank">Getting VsClojure Up and Running in Visual Studio 2010</a></em></p>
<p><a href="http://lh6.ggpht.com/-tupex87Fvwg/TvIVHLCfctI/AAAAAAAAAnE/PglGosORydk/s1600-h/image12.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-hKU7SH-0Mbo/TvIVHaOYqvI/AAAAAAAAAnM/TOGVGlg5Vd8/image_thumb6.png?imgmax=800" width="590" height="326" /></a></p>
<p>After inspecting all of the parameters I noticed that the last one always had a huge number in it as if it was a memory address when integers were the parameters.  When I added an extra parameter to the method declaration and the – function in the in the clj file the calculation returned correctly.    </p>
<p><em>The work-around is only for ClojureCLR 1.3 this has already been fixed in the 1.4 snapshot.  Thank you  to David Miller – he had it fixed in less than 24 hours of my reporting it!</em></p>
<p>In addition to the –ba and –standings functions I have a –main function which will be called when I run the executable. I compile it</p>
<pre>clojure.compile one</pre>
<p>Then run it:</p>
<pre>one.exe </pre>
<p>Gives me the results below</p>
<p><a href="http://lh3.ggpht.com/-q55GtvZYGxs/TvIVHyt9DtI/AAAAAAAAAnU/QLJqI7Z25mE/s1600-h/image4.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-O48QdufzmRQ/TvIVIAEvT4I/AAAAAAAAAnc/4veqjoX8Gto/image_thumb2.png?imgmax=800" width="621" height="192" /></a></p>
<p>In my github project I have a <a href="https://github.com/rippinrobr/clojure-clr-intro/blob/master/3-calling-clojure-from-c-sharp/clj/one14.clj" target="_blank">one14.clj</a> file that works perfectly with the 1.4-snapshot that contains the fix for the issue. It is basically the same as one.clj except it does not need the dummy parameter.</p>
<p>That’s it for the Clojure side of things.  Now its time to talk about the C# code</p>
<h4>The C# Code</h4>
<p>The C# code itself doesn’t have anything remarkable.  It is vanilla C# code.  Which is exactly the way I like it, no special hoops to jump through if I want to call Clojure functions.  In order to call the Clojure code I needed to add a few references.  First I added the assemblies generated when I compiled one.clj: one.clj.dll and one.exe files.  Next I added the assembly Clojure.dll which can be found in the directory where you installed ClojureCLR. After getting the references in place I started writing code.  When I ran the code for the first time I received a System.TypeInitializationException with the message “The type <a href="http://lh6.ggpht.com/-9vTso9fyWoM/TvIVIcfyg3I/AAAAAAAAAnk/CVUaT9HOtik/s1600-h/image2.png"><img style="background-image: none; border-right-width: 0px; margin: 10px 10px 0px 0px; padding-left: 0px; padding-right: 0px; display: inline; float: left; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" align="left" src="http://lh3.ggpht.com/-c1ac5STOHUk/TvIVIx2J91I/AAAAAAAAAns/fMUk7Z2iER4/image_thumb.png?imgmax=800" width="215" height="207" /></a>initializer for 'one' threw an exception." which wasn’t all that helpful.  I dug into the inner exception’s inner exception and found this message: "Could not locate clojure.core.clj.dll or clojure/core.clj on load path." Now that is a message I can do something with.  I added the clojure.core.clj.dll reference and re-ran the app only to find out I was missing a few other references. When I got my code to run I had added the references that are listed on the left.  Once I had the one.clj.dll, one.exe references with the clojure related ones on the left I was ready to go!  In the future adding the Clojure references will be much easier. </p>
<p>Now I can call the ba and standings methods from my C# app just like I call any other .NET static methods.  Here’s the code for calling the Clojure compiled with the 1.3 version compiler.</p><br /><script src="https://gist.github.com/1503170.js?file=gistfile1.cs"></script>
<p>Notice that the extra parameter is the first parameter in the calls from C# even though in the clojure code it is the last parameter.  Here’s what the output looks like for the 1.3 code:</p>
<p><a href="http://lh3.ggpht.com/-nYAhVBiFKsE/TvIVJDigDrI/AAAAAAAAAn0/Z89DFs4D61M/s1600-h/image91.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-YtDPJ4cR6eQ/TvIVJqtv9zI/AAAAAAAAAn8/9prxqhO0GE0/image_thumb5.png?imgmax=800" width="591" height="175" /></a></p>
<p>And the output of the 1.4 snapshot version:</p>
<p><a href="http://lh3.ggpht.com/-liNQVU_w8_o/TvIVJ2HpP8I/AAAAAAAAAoE/CRlPLb3o9qc/s1600-h/image13.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-3MjjVt5vMIY/TvIVKPerwQI/AAAAAAAAAoM/HUqe5XqjWKQ/image_thumb7.png?imgmax=800" width="603" height="164" /></a></p>
<p>Obviously they both produce the same outputs but 1.3 needs the placeholder parameter whereas the 1.4-snapshot version does not.</p>
<h4>Summary</h4>
<p>As you can see there isn’t much work to be done if you want to expose ClojureCLR functions to the outside .NET community.  You just need to tell gen-class which methods to expose and make sure you follow the naming convention for the functions on the clojure side.  On the C# side once you have all the references in place, which will become easier to do in the future, writing the code to call the clojure generated methods is no different than calling any other method in a .NET library.  </p>
<p>If you have any questions or blog post suggestions please feel free to leave a comment.</p>
<h4>Resources</h4>
<p>ClojureCLR: <a href="http://www.myclojureadventure.com/2011/10/getting-started-with-clojure-clr.html" target="_blank">Getting Started with Clojure-clr</a> and <em><a href="http://www.myclojureadventure.com/2011/03/getting-vsclojure-up-and-running-in.html" target="_blank">Getting VsClojure Up and Running in Visual Studio 2010</a>.</em> </p>
<p>C# IDEs: <a href="http://www.microsoft.com/visualstudio/en-us/products/2010-editions/visual-csharp-express" target="_blank">Visual C# 2010 Express</a> or <a href="http://monodevelop.com/Download" target="_blank">MonoDevelop</a></p>
<p>My Source (this blog’s code is the 3-calling-clojure-from-c-sharp): <a title="https://github.com/rippinrobr/clojure-clr-intro/zipball/master" href="https://github.com/rippinrobr/clojure-clr-intro/zipball/master">https://github.com/rippinrobr/clojure-clr-intro/zipball/master</a></p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com3tag:blogger.com,1999:blog-724993853864308794.post-43329473867078262172011-11-30T17:46:00.001-05:002011-12-06T11:30:14.833-05:00Intro to Clojure-clr: Connecting to SQL Server and MySQL<p>Today’s post is a quick introduction to how connect and retrieve rows from a SQL Server and MySQL databases.</p> <h4>The Setup</h4> <p>If you haven’t already done so <a href="http://rob-rowe.blogspot.com/2011/10/getting-started-with-clojure-clr.html" target="_blank">setup Clojure-clr</a>. Obviously you’ll need to have access to a SQL Server and MySQL instances. If you do not have one or both of the databases you can download and install them from here: <a href="http://www.microsoft.com/sqlserver/en/us/editions/express.aspx" target="_blank">SQL Server Express</a>, <a href="http://dev.mysql.com/downloads/installer/" target="_blank">MySQL</a>. After downloading and installing the databases grab the data and schema for this post from here: <a href="https://github.com/downloads/rippinrobr/clojure-clr-intro/ms-sql-battingpost-script.zip" target="_blank">SQL Server version</a>, <a href="https://github.com/downloads/rippinrobr/clojure-clr-intro/battingpost_mysql.zip" target="_blank">MySQL version</a>. Next load the data/schema files then you are ready to go!</p> <h4>Connecting to SQL Server</h4> <p>Since we are working with Clojure-clr I think It would only be proper to start off with connecting to SQL Server. In order to do that I need to load the System.Data assembly. It contains the necessary classes for interacting with the database.</p><script src="https://gist.github.com/1411219.js?file=gistfile1.clj"></script> <p>Once I have System.Data loaded its time to start the connection process. The first thing I need to do is create a connection and then open it.</p><script src="https://gist.github.com/1411285.js?file=gistfile1.clj"></script> <p>Now that I have an open connection I can create a SqlCommand object. The SqlCommand constructor version I’m using takes two parameters a SQL Statement and a database connection object. After creating the SqlCommand object I run SQL statement by calling the ExecuteReader method. Now the data is ready to be retrieved. </p><script src="https://gist.github.com/1438293.js?file=gistfile1.clj"></script> <p>To keep the blog post simple I’m going to grab the results and print out the player id value using a while loop. When the while loop completes I close the reader and the database connection objects.</p><script src="https://gist.github.com/1411325.js?file=gistfile1.clj"></script> <p>That is the quick and dirty way to connect to SQL Server from Clojure-clr. It isn’t as elegant as it is in the JVM version but I hope to get it that way some day. I am working on a project at work that, as time permits, I am <strong>attempting</strong> to port the java.jdbc code over to the CLR. If/When I get it working I will be sure to blog about it. Now it is time to connect to a MySQL database.</p> <h4>Connecting to MySQL</h4> <p>Connecting to MySQL follows the same process as connecting to SQL Server. In fact if you don’t look too closely you might think I’m using the same code to connect to MySQL. I wish that was the case but its not. In order to connect to MySQL from .NET you’ll need to <a href="http://dev.mysql.com/downloads/connector/net/" target="_blank">download the assembly Mysql.Data</a> from the MySQL developer site. I have used the Mysql.Data assembly for awhile so I added it to the GAC which allows me to load it into my Clojure code the same way I did for the SQL Server version. If you don’t want to add Mysql.Data to the GAC you can load it using the assembly-load-from function: (assembly-load-from “the path to the dll”). </p><script src="https://gist.github.com/1411422.js?file=gistfile1.clj"></script> <p>I now have the MySQL libraries loaded and I’m ready to grab the playerId’s from the database. Here is the my-run-it function:</p><script src="https://gist.github.com/1411441.js?file=gistfile1.clj"></script> <p>As you can see the method names are almost the same, everything is prefaced with My. You still do the same process, create the connection, the command, execute the command and read from the reader.</p> <h4>Summary</h4> <p>Connecting to SQL Server and MySQL is pretty straight forward. Just load the appropriate assembly and you are off. In the future I hope to have a cleaner method of interacting with databases through either a direct port of java.jdbc or something very similar. </p> <p>I am relatively new to Clojure so if you see code that I’ve written that makes you cringe please feel free to leave a comment with your suggestion. I am all ears.</p> <h4>About the Data</h4> <p>The data I’m using for this blog post comes from the <a href="http://www.baseball-databank.org/" target="_blank">Baseball Databank</a> project. The project has gathered all the baseball status from previous season and offers the data in many different formats. </p> <h4>Resources</h4> <p><strong>Clojure-clr Setup</strong>: <a href="https://github.com/richhickey/clojure-clr/downloads">Clojure-clr Download Page</a> and <a href="http://rob-rowe.blogspot.com/2011/10/getting-started-with-clojure-clr.html">Getting Started with Clojure-clr</a>.</p> <p><strong>Database Downloads</strong>: <a href="http://www.microsoft.com/sqlserver/en/us/editions/express.aspx">SQL Server Express</a>, <a href="http://dev.mysql.com/downloads/installer/">MySQL</a></p> <p><strong>Data</strong>: <a href="https://github.com/downloads/rippinrobr/clojure-clr-intro/ms-sql-battingpost-script.zip">SQL Server version</a>, <a href="https://github.com/downloads/rippinrobr/clojure-clr-intro/battingpost_mysql.zip">MySQL version</a></p> <p><strong>MySQL Assembly</strong>: <a href="http://dev.mysql.com/downloads/connector/net/">Mysql.Data</a></p> <p><strong>My Source</strong>: <a title="https://github.com/rippinrobr/clojure-clr-intro/zipball/master" href="https://github.com/rippinrobr/clojure-clr-intro/zipball/master">https://github.com/rippinrobr/clojure-clr-intro/zipball/master</a></p> Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com13tag:blogger.com,1999:blog-724993853864308794.post-74974261415458139732011-11-22T11:01:00.001-05:002011-11-22T11:14:54.667-05:00Intro to Clojure-clr: Using the spit function<p>A couple of weeks back I opened an issue with the Clojure-clr ‘team’ because I thought the spit function had a bug.   When I checked my email this morning I had a response from David Miller who runs the Clojure-clr port. In the email he politely showed me the error of my ways. I wanted to share this with anyone else who may have been looking to use spit to append to a file on the CLR. </p> <h4>Using spit the JVM way</h4> <p>When I was trying to see if I could use spit to append to a file I did a quick Google search and found this page on the ClojureDocs site for <a href="http://clojuredocs.org/clojure_core/1.3.0/clojure.core/spit">spit</a>. When I saw their append example below I tried it in the Clojure-clr REPL.</p> <p><a href="http://lh4.ggpht.com/-DCUAg9NaT5E/TsvHXf4DWmI/AAAAAAAAAms/8ZkG3TDnqXE/s1600-h/image%25255B7%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-8mmYlv_Y8EU/TsvHX5f204I/AAAAAAAAAm0/LxSgria1fJA/image_thumb%25255B3%25255D.png?imgmax=800" width="610" height="176" /></a></p> <p>I received no errors from the spit function but as you can see each call to the function was overwriting the previous call.  My next step was to try the same thing in the JVM REPL to see if it worked as the web page stated it should.  On the JVM spit worked as advertised.  Up to this point I hadn’t come across a core function on the CLR that didn’t have the same type of parameters as the JVM function. This lead me to believe I had found a bug so I submitted a bug report.</p>
<h4>Using spit the CLR way</h4>
<p>Fast forward a couple of weeks to the point where I checked my email this morning. In his response David laid out what I had done wrong and gave more information related to spit:</p> <pre>Lack of documentation is definitely an issue here.<br /><br />Clojure 1.4.0-master-SNAPSHOT<br />user=> (spit "hi.txt" "Test 1\n" :file-mode System.IO.FileMode/Append)<br />nil<br />user=> (spit "hi.txt" "Test 2\n" :file-mode System.IO.FileMode/Append)<br />nil<br />user=> (println (slurp "hi.txt"))<br />WARNING: (slurp f enc) is deprecated, use (slurp f :encoding enc).<br />Test 1<br />Test 2<br /><br />Generally, the options available are ones that can be handled by the appropriate methods/ctors in System.IO.<br /><br />In the source:<br /><br />Common options include<br /><br /> :buffer-size Ths size of buffer to use (default: 1024).<br /> :file-share A value from the System.IO.FileShare enumeration.<br /> :file-mode A value from the System.IO.FileMode enumeration.<br /> :file-access A value from the System.IO.FileAccess enumeration.<br /> :file-options A value from the System.IO.FileOptions enumeration.<br /> :encoding The encoding to use, either as a string, e.g. \"UTF-8\",<br /> a keyword, e.g. :utf-8, or a an System.Text.Encoding instance,<br /> e.g., (System.Text.UTF8Encoding.)<br />More documentation is most certainly needed.</pre>
<p>I gave it a try on my 1.3 REPL with the System.IO.FileMode/Append and it worked like a champ!  </p>
<h4>Lessons Learned</h4>
<p>So what did I learn from this?  Next time I come across something that I think is a bug I will check the Clojure-clr source first.  It is ok to write lazy code but the programmer himself cannot be lazy!</p>
<h4>Summary</h4>
<p>The CLR version of the spit function differs slightly from the JVM version.  We can use the :file-mode key with the value System.IO.FileMode/Append when we want to append to a file using spit.  It is also a good idea to check the Clojure-clr source prior to reporting any other issues I come across.  Thank you David for your kind response!</p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com0tag:blogger.com,1999:blog-724993853864308794.post-51567035515410682122011-11-22T08:58:00.001-05:002011-11-22T11:04:35.681-05:00Hugo-clr: Parsing Web Pages with Clojure-clr and HtmlAgilityPack<p>
When I first became interested in learning Clojure I was in the middle of a science fiction reading kick and I was looking for new authors to read. So I decided I would try and pick up Clojure by writing code to parse the winners and nominees for the Best Novel category on the <a href="http://www.thehugoawards.org/">Hugo Awards</a> web site.  While I was writing the code writing I decided I would share my experience as a Clojure noob (still am) through a three part blog series that covered what I did with Clojure on the JVM (<a href="http://rob-rowe.blogspot.com/2011/05/parsing-web-pages-with-clojure-and.html,">Parsing Web Pages with Clojure and Enlive</a>, <a href="http://rob-rowe.blogspot.com/2011/05/creating-hugo-awards-db-with-clojure.html">Creating a Hugo Award DB with Clojure and Sqlite</a>, and <a href="http://rob-rowe.blogspot.com/2011/06/creating-simple-ui-for-hugo-db.html">Creating a Simple UI for the Hugo DB</a>) .  </p>
<p>
About a month ago I decided to really give Clojure-clr a try so I thought I would go through the same process I did on the JVM version. Why? I thought it would give me a good way to compare and contrast the JVM and CLR versions of Clojure. Not that I’m a Clojure guru, I’m new to the world of parenthesis but doing the same project will allow me to point out the differences I came across between the CLR and JVM versions.  With that said, let’s make sure you have your Clojure-clr environment set up.</p>
<h3>
Setup</h3>
<p>
Since the CLR world doesn’t have lein or a lein equivalent I have to do the configuration by hand. The first step is to install Clojure-clr if you haven’t already installed it.  My post <a href="http://rob-rowe.blogspot.com/2011/10/getting-started-with-clojure-clr.html">Getting Started with Clojure-clr</a> will walk you through the steps. After setting up Clojure-clr download the <a href="http://htmlagilitypack.codeplex.com/">HtmlAgilityPack</a>.  It is the .NET library I am using to parse the Hugo web pages.  If you want the HtmlAgilityPack lib and source code you can grab it here: <a href="https://github.com/rippinrobr/hugo-clr/tree/hugoclr-parser">https://github.com/rippinrobr/hugo-clr/tree/hugoclr-parser</a> and follow along that way.  Just make sure you have the code from the hugoclr-parser branch.  With the setup complete it is time to start looking at some code.</p>
<h4>
hugoclr.clj</h4>
<p>
The hugoclr.clj file is where the –main function lives. It calls hugoclr.parser/get-awards to retrieve the award pages, parse the nominees and winners data out and  passes the results to the hugoclr.data.csv/write-to-file function to write out the data in a comma-delimited file. </p>
<script src="https://gist.github.com/1371981.js?file=hugoclr.clj"></script> <p>
There are only a couple of items I’d like to point out in the hugoclr.clj file. First is the way that the HtmlAgilityPack library is loaded.</p>
<pre>(assembly-load-form "..\\libs\\HmtlAgilityPack.dll")</pre>
<p>
The function assembly-load-from is a new function to Clojure-clr.  It was added in the 1.3 release.  It is a wrapper around the System.Reflection.Assembly/LoadFile call. I find the assembly-load-form more clojure’esque and less typing so I’ve started using it.  </p>
<p>
The next line of interest is the :gen-class line.  Using the :gen-class call is what triggers the generation of the hugoclr.exe file. if I didn’t add that line to my source I would only generate DLLs when I compile hugoclr.  That’s it for hugoclr.clj. Its main purpose in life is to kick off the parsing and pass the results to the hugoclr.data.csv/write-to-file function. Next, I’ll discuss work horse of the project, the hugoclr/parser.clj file.</p>
<h4>
hugoclr/parser.clj</h4>
<p>
The hugoclr/parser.clj file is where most of the work is done. It handles the fetching of the web pages, parsing the award page links, and grabs the data from the awards pages, and converts the data into records that can will be used later. The entry point into the file is the get-awards function.</p>
<h5>
get-awards / get-html-elements / fetch-url</h5>
<script src="https://gist.github.com/1374261.js?file=gistfile1.clj"></script>
<p>
The get-awards function is the ‘main’ function of the hugoclr/parser.clj file. It is what drives the parsing process. The function starts by calls the get-html-elements function passing a URL to the history page and the XPATH that when applied will return a sequence of anchor tags starting with the 2011 awards page link. </p>
<p>
Next get-html-elements passes the URL to fetch-url.  fetch-url makes a request to the URL by creating a HtmlAgilityPack.HtmlWeb object and making a call to the HtmlWeb.Load method.  The HtmlWeb.Load  method ‘converts’ the retrieved web page into a HtmlDocument object. </p>
<p>
The returned HtmlDocument’s SelectNodes method is called the XPATH that was passed to get-html-elements. SelectNodes applies the XPATH and returns a sequence of HtmlNode objects that represent the anchor tags on the Hugo History page. Since I only want the anchor tags that will lead me to the awards pages I us the map function to pass the HtmlNode objects through the validate-award-link function.  The results of the map call is a sequence of links to award pages or nulls.  The nulls are in the place of links that were not award page links. I remove them by calling filter passing a function that only keeps non-null entries. At the end of this process I have a sequence of valid award page links.  </p>
<p>
The last step of collecting the nominees and winners data is to parse each individual award page.  I start by taking the first 12 links from the awards-link sequence and pass each one to the parse-awards-page function using the map function.  Each link is then processed in the parse-awards-page function returning a sequence of Category records that represent each awards category for the given year.  Now I have a sequence of Category sequences ready to be written out to a file.  Before I go over that part of the code I would like to walk you through the parse-awards-page function. </p>
<p>
<em>You may be asking yourself why I’m only taking the 2000s.  The answer is simple, I’m lazy.  While writing the JVM and CLR versions I found that if I didn’t load the pages first in a browser I was unable to retrieve them programmatically.  So if I wanted to process all of the pages I would have had to load them all.  I’d be bored before I got of the 90s so I cut it off at 2000.  </em></p>
<h5>
parse-awards-page</h5>
<script src="https://gist.github.com/1377924.js?file=gistfile1.clj"></script>
<p>
parse-awards-page uses the get-html-elements function to get a HtmlDocument object that represents the awards page to parse.  The function then passes the object to the create-category-record function which as you might expect creates a Category record that represents each award category on the awards page.  Since each page has more than one award category parse-awards-page returns a sequence of Category records.  </p>
<h5>
create-category-record</h5>
<script src="https://gist.github.com/1380444.js?file=gistfile1.clj"></script>
<p>
As I said earlier, the Category record is the data structure that represents the nominees and winners of a particular Hugo Award category.  The first step in creating a Category record is to find the paragraph tag that appears just before the category’s UL tag.  The paragraph tag contains the year the award was given and the name of the award.      </p>
<p>
Once I have the paragraph node the next step is to find all of the list item tags in the award category’s unordered list.  All but the first of the li tags contain the text that describe the nominees and winners for the award category currently being parsed. The nominee/winner li nodes are passed through a filter to make sure that only the li tags are kept.  </p>
<p>
Now that I have the paragraph and li tags I’m ready to create the Category record.  The get-category-heading and get-year functions simply parse the text from the paragraph tag and return the award name and year.  The li tags are passed to the create-works-seq function which creates a sequence of Work records that represents nominees and winners for the category.   Once each category on the page has been parsed control is returned back to the parse-awards page so it can continue parsing the award pages until they have all been processed.</p>
<h4>
A Quick Side Note: Records vs. Structs</h4>
<p>
When I wrote the JVM version of this ‘application’ I used structs to model the categories and works.  Using structs worked fine for what I was doing.  However when I started writing the CLR project I was in the middle of reading the book <a href="http://www.amazon.com/gp/product/1935182641/ref=as_li_ss_tl?ie=UTF8&tag=rrowecom-20&linkCode=as2&camp=217145&creative=399369&creativeASIN=1935182641">The Joy of Clojure: Thinking the Clojure Way</a><img style="border-bottom-style: none !important; margin: 0px; border-left-style: none !important; border-top-style: none !important; border-right-style: none !important" border="0" alt="" src="http://www.assoc-amazon.com/e/ir?t=rrowecom-20&l=as2&o=1&a=1935182641&camp=217145&creative=399369" width="1" height="1" /> by Michael Fogus and Chris Houser.  The authors mentioned that records have some advantages over structs and for that reason structs are falling out of favor.  Some of the advantages of records are that the are created quicker than structs and take up less memory.  They also look up keys quicker than array or hash maps.  After reading that I went with records instead of structs in the CLR version.  By the way I have really enjoyed reading <a href="http://www.amazon.com/gp/product/1935182641/ref=as_li_ss_tl?ie=UTF8&tag=rrowecom-20&linkCode=as2&camp=217145&creative=399369&creativeASIN=1935182641">The Joy of Clojure</a> and I would highly recommend it.  </p>
<h4>
And Now Back to the Code…</h4>
<p>
Now that we have parsed the all of 2000s award pages the only step left is to write the results out to a comma-delimited text file.  In the –main function the results of the get-awards are passed to hugoclr.data.csv/write-to-file as its first parameter and the name of the output file as its second parameter.  Lets walk through the last bit of code, the hugoclr/data/csv.clj file.</p>
<h5>
hugoclr.data.csv.clj</h5>
<p>
The write-to-file method does exactly what its name implies, writes something to a file.  In our case it takes the awards, converts each record into a comma-delimited line and then writes them to the output file.</p>
<script src="https://gist.github.com/1380790.js?file=gistfile1.clj"></script>
<p>
First I create a writable stream using .NET’s System.IO.StreamWriter class.  I’ve told the stream to write the results to c:\temp\hugo.txt.  I could have used the spit function but I decided to use a .NET library here.  Once I have the stream I pass each category to the delimit function which simply cleans the title and publisher string and places a comma between all of the Work record’s fields. After each category has been converted the lines are then reduced into a single string.  The string is written to the output file.  Running the code produces an output file a file like this:</p>
<script src="https://gist.github.com/1380820.js?file=gistfile1.txt"></script>
<h4>
Running hugoclr</h4>
<p>
Now that I’ve walked you through the guts of the code it is time to show you what it looks like when it runs.  First, I will show you how to run it in the REPL.</p>
<p>
<a href="http://lh5.ggpht.com/-j_a8ZgxZ_uU/Tsuqlvj7uyI/AAAAAAAAAmM/ETd8wK8AoQA/s1600-h/image%25255B4%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; margin-left: auto; border-left-width: 0px; margin-right: auto; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-461kZMwe3yE/TsuqmJF-ivI/AAAAAAAAAmU/r4wn_x28eBk/image_thumb%25255B2%25255D.png?imgmax=800" width="557" height="357" /></a></p>
<p>
It is pretty straight forward.  Fire up the REPL, load the hugoclr.clj file and then call the –main function.  From there the code grabs the link page, parses it out and lets you know where it is in the process by telling you which page it is retrieving.  </p>
<p>
<em>Remember, you must ‘prime’ the app before you run the code by loading each page in your favorite browser. I’m not sure why this is required. If anyone knows why this is happening and knows a way around please let me know.</em></p>
<p>
Next, I will compile and run the code from the command line.</p>
<p>
<a href="http://lh5.ggpht.com/-UZo2f9wn8aE/TsuqmtxTYdI/AAAAAAAAAmc/qnmpoNX3AeY/s1600-h/image%25255B9%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; margin-left: auto; border-left-width: 0px; margin-right: auto; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-LcTzJbZYLEw/Tsuqmy_IYJI/AAAAAAAAAmk/beWNelf5eso/image_thumb%25255B5%25255D.png?imgmax=800" width="574" height="346" /></a></p>
<p>
One thing to keep in mind when you compile your CLR code with Clojure 1.3.0 Debug on the 4.0 .NET CLR the executable and DLLs generated are placed in the compiler’s directory.  Obviously the results are the same either way I run it.</p>
<h4>
Summary</h4>
<p>
Parsing the Hugo Awards list for the winners in the 2000s wasn’t all that different from the JVM version.  I did find using the HtmlAgilityPack library a little easier to work with when parsing the web pages. This probably due to my familiarity with HtmlAgilityPack since I’ve used it in a few C# projects.  Another reason I found it easier this time around is probably related to the fact that I’m a ‘little’ more comfortable writing Clojure code.  I still have a long way to go though before I’m fluent in it. </p>
<p>
Writing Clojure in the CLR environment wasn’t much different in this part of the project than the JVM version.  In the CLR world we don’t have things like lein but so far I haven’t come across any issues that would prevent me from continuing to become familiar with Clojure CLR in hopes of using it at my day job.  Which may come soon as in the next few days.</p>
<p>
My next post in this serious will be on taking the data from the csv file, creating a SQL Server table, and loading the new table with the data from the file.</p>
<p>
Since I am still pretty green in the Clojure world please feel free to leave a comment if you see something that is not idiomatic Clojure or if there is a better way to do something.  I’m eager for any and all feedback.</p>
<h4>
Resources</h4>
<p>
<a href="https://github.com/richhickey/clojure-clr">Clojure-clr</a> I’m using the 1.3 version with .Net 4.0 and <a href="http://htmlagilitypack.codeplex.com/#">HtmlAgilityPack</a> <br/>
<a href="http://www.amazon.com/gp/product/1935182641/ref=as_li_ss_tl?ie=UTF8&tag=rrowecom-20&linkCode=as2&camp=217145&creative=399369&creativeASIN=1935182641">The Joy of Clojure: Thinking the Clojure Way</a></p>
<h4>
The Code</h4>
<p>
,You can download the code for this post from <a href="https://github.com/rippinrobr/hugo-clr/tree/hugoclr-parser">https://github.com/rippinrobr/hugo-clr/tree/hugoclr-parser</a> .  Just make sure you are on the hugoclr-parser branch.</p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com0tag:blogger.com,1999:blog-724993853864308794.post-91379574215466709942011-11-04T12:11:00.000-04:002011-11-04T12:11:17.983-04:00Intro to Clojure-clr: How to Interact with .Net objects<p>Today’s blog post is another ‘quick hitter’ covering how to instantiate a .NET object and interact with it’s instance methods and properties.</p><h4>The Setup</h4><p>If you haven’t already installed Clojure-clr take a look at my previous post:  <a href="http://rob-rowe.blogspot.com/2011/10/getting-started-with-clojure-clr.html">Getting Started with Clojure-clr</a> and get it installed.  Next, open up a cmd.exe session and enter clojure.main.exe. to start a REPL session.  Throughout all of my Clojure-clr blog posts I will assume you have added the clojure-clr directory to your PATH variable.</p><h4></h4><h4>Instantiating a .NET Object</h4><p>According to the <a href="https://github.com/richhickey/clojure-clr/wiki/CLR-Interop">CLR Interop page</a> there are two ways to instantiate an object:</p><p>(Classname. args*) or (new Classname args*)</p><p>I will use the first function to create a System.IO.StreamWriter object that I will use to write to a file.  At the REPL prompt enter:</p><p>(def file (System.IO.StreamWriter. “testing.txt”))</p><p>The line above created a StreamWriter object that will write to a file named testing.txt.  Testing.txt will be in the directory where we started the REPL.  The object is stored in the symbol named file. Now it's time to write a line to testing.txt.</p><h4>Calling an Instance Method StreamWriter.WriteLine</h4><p>To write our line enter the following lines into the REPL: </p><p>(.WriteLine file “This is a line sent from the REPL!”)</p><p>(.Close file)</p><p>Since WriteLine isn’t a static method we start off the list with the method call with .WriteLine. Next comes file, the symbol that represents our StreamWriter object. Everything after file is a parameter passed to WriteLine method, in this case it is the string I want to write to the output file.  The .Close call flushes the buffer and closes the file.  Ok, I’ve put the two lines into the REPL and ran them.  How do I know if the line was actually written?  Run the line below in the REPL:</p><p>(println (slurp “testing.txt”))</p><p>The slurp function reads and returns the contents of the testing.txt file. The println function prints it out to the REPL screen.  Your REPL should look something like this:</p><p><a href="http://lh5.ggpht.com/-jhJ8dmIGdxA/TrQLwB04YHI/AAAAAAAAAl8/iCcrRUYOVXY/s1600-h/image%25255B3%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-BYoEEb6JRX0/TrQLwco-O0I/AAAAAAAAAmE/-z5fr9nBcWs/image_thumb%25255B1%25255D.png?imgmax=800" width="615" height="200" /></a></p><p>I can see that my code did write out the line like I had hoped.  However the slurp function generated a deprecated message.  It is an easy issue to resolve by adding :encoding “ascii” to the call. The deprecation message only appears in the CLR REPL.  If I run the same line in the JVM REPL I don’t see this message.  So when I’m using slurp in CLR land I call slurp like this:</p><p>(slurp f :encoding “ascii”)</p><p>Now when I run slurp I see the line from testing.txt without the deprecation message.  If you want to see a list of all the possible encoding values check out <a href="https://github.com/richhickey/clojure-clr/blob/master/Clojure/Clojure.Source/clojure/clr/io.clj">io.clj in the Clojure-clr source repo</a>.  The encoding values start on line 179.</p><h4>Summary</h4><p>You should now be able to instantiate a .NET object and call an instance method.  In this case I created a StreamWriter object and used it to write a line out to a file.  During the process we did see a difference between the JVM version of slurp and the CLR version of slurp which was easy to resolve the issue by passing in :encoding “ascii”.  </p><h4>Resources</h4><p><a href="https://github.com/richhickey/clojure-clr/downloads">Clojure-clr download page</a>,  <a href="https://github.com/richhickey/clojure-clr/wiki/CLR-Interop">Clojure-clr Interop page</a>, <a href="http://rob-rowe.blogspot.com/2011/10/getting-started-with-clojure-clr.html">Getting Started with Clojure-clr</a>, <a href="https://github.com/rippinrobr/clojure-clr-intro/blob/master/1-calling-a-dotnet-method/method.clj">An example that parses a web page and writes to a file using .NET objects</a></p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com0tag:blogger.com,1999:blog-724993853864308794.post-40075353376314445942011-10-27T09:04:00.000-04:002011-12-21T13:05:37.857-05:00Getting Started with Clojure-clr<p><i><b>Update:</b> All github ClojureCLR links have been updated to use the new repository at <a href="https://github.com/clojure/clojure-clr/">github.com/clojure/clojure-clr</a></i></p>
<p>After taking the summer off from blogging to spend time with the family, I am ready to get back to my blog. My first post of the ‘fall blogging season’ is a brief intro Clojure-clr. In this post I will walk you though getting clojure-clr installed, show you how to interact with the REPL and compile an application.</p><h4>The Goal</h4><p>By the end of this post I will be able to interact with .NET assemblies from the clojure-clr REPL and be able to compile a basic clojure-clr application from the command line.</p><h4>The Setup</h4><p>The first thing to do is to visit the <a href="https://github.com/clojure/clojure-clr/downloads" target="_blank">Download Page</a> of the <a href="https://github.com/clojure/clojure-clr" target="_blank">Clojure-clr project on github</a>. I chose to install the clojure-clr-1.3.0-Debug-4.0zip version. Once you have downloaded a version follow these instructions <a href="https://github.com/clojure/clojure-clr/wiki/Getting-binaries" target="_blank">Getting started binary distribution page</a>. Make sure to ‘Unblock’ the zip file before you unzip it. Failure to do so will cause exceptions to be thrown every time you try to run any of the executables or use any of the DLLs in the zip file. For ease of use I added the directory I extracted the files to into my PATH environment variable.</p><h4>The REPL</h4><p>Next, I started up cmd.exe and typed: <em>clojure.main.exe . </em> This will start up the REPL. If everything goes right you should see something similar to this image below. </p><p><a href="http://lh4.ggpht.com/-wWC9cuu-S8M/TqlR4C7Co5I/AAAAAAAAAkc/FhhGuTb3fuc/s1600-h/image3.png"><img style="background-image: none; border-right-width: 0px; margin: 0px auto; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-EuJykqK5nQ8/TqlR4uzDK7I/AAAAAAAAAkk/SkB5Wg79PIU/image_thumb1.png?imgmax=800" width="308" height="95" /></a></p><p>The version number you see will be determined by the version you downloaded. Now that we have the REPL up and running lets take try some clojure code.  At the prompt enter:  (println “yep, it worked!”)  and press the enter key.  That should give you:</p><p><a href="http://lh4.ggpht.com/-LTAWfW4aGZ8/TqlR4y-zTuI/AAAAAAAAAks/hdhL6LKiudw/s1600-h/image7.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; margin-left: auto; border-left-width: 0px; margin-right: auto; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-9f-LBBxBlTk/TqlR5OZTz9I/AAAAAAAAAk0/4j1Tos03XlA/image_thumb3.png?imgmax=800" width="314" height="133" /></a></p><p>Ok, the ‘normal’ clojure seems to be working, now its time to try interacting with .NET.</p><h4></h4><h5>Interacting with .NET</h5><p>Now that we know we have the REPL working it is time to try out a few calls into the .NET world.  The first call we will make is to Console.WriteLine.  Since System.Console is loaded automatically by the REPL we can call it like this:  </p><p>(System.Console/WriteLine “I just called a .NET method!”)</p><p><a href="http://lh5.ggpht.com/-JBjfxZoWnUE/TqlR5eoW59I/AAAAAAAAAk8/lDbWmFtuTWU/s1600-h/image11.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-LgJUzJZEDTE/TqlR56jo4kI/AAAAAAAAAlE/1IjCyRZTYh0/image_thumb5.png?imgmax=800" width="591" height="92" /></a></p><p>It works the same as the println call did.  What if you want to interact with a .NET lib that hasn’t already been loaded into our namespace by the REPL?  There are a couple of ways to do it one takes a lot of typing and the other takes about half the amount.  To illustrate I will load the System.Windows.Forms assembly and load the MessageBox class. The verbose way to load a library is to call the Assembly.Load method:</p><p>(System.Reflection.Assembly/Load "System.Windows.Forms, <br />
Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089")</p><p>When you hit enter after typing the above code in you should see this:</p><p><a href="http://lh5.ggpht.com/-b7mcX3A-Xus/TqlR6dZauwI/AAAAAAAAAlM/0HfdL5mSWf0/s1600-h/image16.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-0DzGpsJLOLU/TqlR6nmVIoI/AAAAAAAAAlU/UzymraF3rZE/image_thumb8.png?imgmax=800" width="582" height="94" /></a></p><p>The second way to load a .NET assembly takes far less typing.  It makes use of the Assembly/LoadWithPartialName method.  Here’s what it looks like:</p><p>(System.Reflection.Assembly/LoadWithPartialName "System.Windows.Forms")</p><p>The results of this call will give you the same output as the more verbose Load call did.  Either way you choose will work fine.  If loading a particular version of a an assembly is crucial then I would go with the Load call.  Otherwise I would stick with the LoadWithPartialName method.  </p><p>Now that we have loaded System.Windows.Forms I’m going to include it into our namespace so I can use the MessageBox class so I can make calls into it a little cleaner.  I can do that by typing the following in the REPL</p><p>(import (System.Windows.Forms MessageBox))</p><p>The call above tells the REPL we want to bring the System.Windows.Forms.MessageBox into our user namespace.  This will allow us to use MessageBox/Show instead of System.Windows.Forms.MessageBox/Show.   When entered the line of code below I saw the dialog box that follows it. Pretty straight forward.</p><p>(MessageBox/Show “Hi from clojure-clr!” “Clojure-CLR Dialog”)</p><p><a href="http://lh5.ggpht.com/-wrpUU37zmk4/TqlR61UAY4I/AAAAAAAAAlc/jUqIXHVhkhU/s1600-h/image19.png"><img style="background-image: none; border-right-width: 0px; margin: 10px auto 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-_MKyuFS9Mn0/TqlR7bse3oI/AAAAAAAAAlk/gz8KC9wENPU/image_thumb9.png?imgmax=800" width="161" height="148" /></a></p><p>Now that I’ve shown you how to make a call into .NET from the REPL let’s compile a clojure-clr ‘application’.  </p><h4>Compiling a Clojure-clr Application</h4><p>For our example I will create an ‘application’ that consists of all the code that we entered into the REPL above in a file called <a href="https://raw.github.com/gist/1309958/1f56b7de6345464e51e88a17691e4c8bb1abc895/intro.clj" target="_blank">intro.clj</a> .  After you have downloaded the file, change into the directory where it was saved.  Then run:</p><p>clojure.compile intro </p><p>The compile command will create the following files in the same directory where the clojure.compile.exe file lives.  <a href="http://lh6.ggpht.com/-6NDa-uAzuVw/TqlR7k3KEJI/AAAAAAAAAls/epzTG3Ad1VM/s1600-h/image23.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; margin-left: auto; border-left-width: 0px; margin-right: auto; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-Gqnha5dODe8/TqlR705xSHI/AAAAAAAAAl0/w9eBIss_N3I/image_thumb11.png?imgmax=800" width="537" height="62" /></a></p><p>I haven’t found out how to tell the compiler where to put the output yet so for now I just run them from that directory.  (<em>If you know how to tell the compiler where to put the output please leave a comment below.  I would really appreciate it!)</em></p><p>Running the intro.exe file will produce the println, Console.Writeln, and MessageBox.Show output just as it was in the REPL.</p><h4>Summary</h4><p>That’s it for my very brief intro into the world of clojure on the CLR.  I walked you through using the REPL to call ‘normal’ clojure and how to interact with .NET.  I also showed you how to compile a clojure-clr application on the command line.</p><p>In future clojure-clr posts I will convert my hugo project posts (<a href="http://rob-rowe.blogspot.com/2011/05/parsing-web-pages-with-clojure-and.html">Parsing Web Pages with Clojure and Enlive</a>, <a href="http://rob-rowe.blogspot.com/2011/05/creating-hugo-awards-db-with-clojure.html">Creating Hugo Awards DB-with Clojure</a>, and <a href="http://rob-rowe.blogspot.com/2011/06/creating-simple-ui-for-hugo-db.html">Creating a Simple UI for the Hugo Awards DB</a>) to hugo-clr using the .NET equivalents.  Plus I plan on having a post or two on how to use the clojure-clr assemblies in a C# application.</p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com15tag:blogger.com,1999:blog-724993853864308794.post-58950819538115856252011-06-15T08:09:00.000-04:002011-06-24T08:47:49.446-04:00Creating a Simple UI for the Hugo DB<p>In my past two posts, <a href="http://rob-rowe.blogspot.com/2011/05/parsing-web-pages-with-clojure-and.html" target="_blank">Parsing Web Pages with Clojure and enlive</a> and <a href="http://rob-rowe.blogspot.com/2011/05/creating-hugo-awards-db-with-clojure.html" target="_blank">Creating a Hugo Awards DB with Clojure and Sqlite</a>, I parsed and stored the list of Hugo Best Novel award nominees and winners. In this, the last post on the Hugo data, I will build a very basic UI in Clojure using Swing and Java’s awt library.</p> <h4>The Goal</h4> <p>By the end of the post I will have code to display a listing of the nominees in a UI. The data will come from the sqlite database I created in my <a href="http://rob-rowe.blogspot.com/2011/05/creating-hugo-awards-db-with-clojure.html" target="_blank">previous post.</a></p> <h4>The Setup</h4>
<p>If you’ve been following the past few posts then you already have everything you need installed. The only thing you will need to do is grab the code from my <a href="https://github.com/rippinrobr/hugo/tree/hugoUI" target="_blank">hugoUI branch</a>.</p>
<p>For those of you who haven’t been following along the first thing you will need to do is grab sqlite. Since I’m working on I windows I grabbed these two downloads: <a href="http://www.sqlite.org/sqlite-shell-win32-x86-3070600.zip">sqlite shell</a> and <a href="http://www.sqlite.org/sqlite-dll-win32-x86-3070600.zip">sqlite-dll</a>. After downloading the files make sure they are in your PATH.</p>
<p>Next, grab <a href="https://github.com/technomancy/leiningen">leiningen</a>. Leiningen is a Clojure build tool that helps you manage your projects. I use in all of my clojure projects. The install takes no time at all, just follow the instructions on the <a href="https://github.com/technomancy/leiningen">project’s page</a> and you will be ready for business.</p>
<h4>Updating the the project.clj file </h4>
<p>Since I am not going to be doing any HTML parsing I've removed the enlive dependency from my project.clj file. All other dependencies I had in the previous version still remain.</p>
<script src="https://gist.github.com/1012428.js?file=project.clj"></script>
<p>Just to be on the safe side I ran lein deps to ensure I have everything ready.</p>
<h4>The Code</h4>
<p>The UI app consists of four code files: cmdline.clj, main_controller.clj, sqlite.clj, and main_view.clj. The cmdline.clj file contains the main method and starts the app. The main_controller.clj file retrieves the data and calls the function that creates the view. The data is retrieved by calling the functions in the sqlite.clj file. The last file is the main_view.clj file who's job is to create the user interface and display the data.</p>
<h5>The cmdline.clj file</h5>
<script src="https://gist.github.com/1012472.js?file=cmdline.clj"></script>
<p>The cmdline.clj file’s sole purpose is to start the application. This is done in part by using the :gen-class macro. The macro will create a java class file called cmdline.class in the project’s classes directory. Any methods in the java class will look in the source clj file for a method by the same name but preceded by a –. That is why the –main function exists in cmdline.clj. It is what is called when I execute the app outside of the REPL. Within the main function I call hugo.controller.main-controller/show-list to kick off the data retrieval and displaying of the UI.</p>
<h5>The hugo.controllers.main-controller.clj file</h5>
<script src="https://gist.github.com/1012517.js?file=main-controller.clj"></script>
<p>The functions in this file are concerned with retrieving the data, formatting it and passing it along to the view.  Before I jump into what the functions are doing I want to point out my use of the defn- macro.  Using this macro allows me to ‘hide’ the function from anyone outside of the namespace that it was created in.  So in the hugo.controllers.main-controller namespace there is only one function that is visible outside of the namespace.  That is the show-list function.</p>
<p>The show-list function is a simple one.  It starts off by calling the get-nominees-data-list function which is where the majority of the data retrieval and formatting takes place.  It starts off by calling the hugo.db.sqlite/get-years function. As you might expect the get-years function retrieves all of the years that are stored in the database.  I use the map function to iterate over the returned sequence of the years.  For each year the format-data function is called.</p> <p>The format-data function takes a year as its only parameter.  The year is then passed to the hugo.db.sqlite/get-nominees function which returns a sequence of maps for all nominees in the year specified.  The results of the get-nominees call are passed through their own map function that is used to format the data into a string that can be displayed in a list. Finally, the results of the get-nominees map file are concat'ed with a title string and a spacing string to create sequence that is ready to be displayed on the UI.  Now that I have all of the data retrieved and formatted its time to display it.</p>
<h5>The hugo.views.main-view file</h5>
<script src="https://gist.github.com/1012623.js?file=main-view.clj"></script>
<p>As I said earlier I am creating a UI using the Java UI libraries Swing and awt.  The :import macro brings in the BorderLayout and ActionListener classes.  The BorderLayout class is used for laying out the objects on the UI.  The ActionListener class is used for the event handler when the Close button is clicked.  The classes I’m bringing in from Swing are all UI objects except for the DefaultListModel class.  That class is used to manage the list of novels that will be displayed in the list box.</p>
<p>The hugo.controllers.main-controller/show-list function calls the create-home-view function after it has retrieved and formatted the data. Inside of create-home-view all of the UI objects are created. First I need to create the main window by creating an instance of the JFrame object passing in the title of the window. Next I create the panel that will contain all of my UI objects using JPanel. The title label and close button objects are pretty self-explanatory. </p>
<p>The last three lines of the let statement are all relate to creating the list. The list-model is used to manage the data in the list. Once I have a list model object I can create the JList object. When the JList object is created I passed in the list-model object to associate the list with the list-model. The last object created is the JScrollPanel object which is used to provide scrolling capabilities for the list box.</p>
<p>Now that I have the UI objects created its time to call the ‘private’ function add-list-items. This function’s job is to add the formatted list data to the list-model object. The add-list-items function uses doseq to loop over the data sequence.  Each item is added to the list-model object using the addElement method.  Basically, the add-list-items takes the items from the data sequence and adds them to the list box via the list-model.</p>
<p>Once the data has been handled its time to dress up the title-label. I can change the look and feel of the label by calling the JLabel class's setBackground, setForeground and setFont methods. I changed the look of the close button by calling the equivalent methods on the JButton class. Since I want something to happen when a user clicks the close button I added the event handler function close-button-handler by calling the addActionListener method.</p>
<p>After the cosmetic steps have been completed it is time to add the objects to the frame and size the main window. The last line of code makes the window visible. Now that I have everything ready its time to run the app. To run the app I used:</p>
<pre>lein run</pre>
<p>When the app comes up it should look like the picture below. Not too bad for my first Clojure UI.</p>
<h3><a href="http://lh3.ggpht.com/-bEcxNzKVu78/Tfekp8oQ_YI/AAAAAAAAAkU/8PIiOfBnvnY/s1600-h/hugo-ui%25255B4%25255D.png"><img style="background-image: none; border-right-width: 0px; margin: 0px 10px 0px 0px; padding-left: 0px; padding-right: 0px; display: inline; float: left; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="hugo-ui" border="0" alt="hugo-ui" align="left" src="http://lh4.ggpht.com/-WdvUb8g2SiM/Tfekq9_FxUI/AAAAAAAAAkY/phC7GzwO9OY/hugo-ui_thumb%25255B2%25255D.png?imgmax=800" width="273" height="410" /></a> Summary</h3>
<p>Creating a user interface in Clojure is not difficult. If someone like me who makes his living in the .NET world can create a UI like this I would imagine it would be much easier for Java devs. As always I’m looking for any and all feedback on my code.  Do not hesitate to leave a comment.</p>
<h3>Help Me Raise Money for The Leukemia and Lymphoma Society!</h3>
<p>This November I will be running my first half marathon. I’m running as a member of the Team in Training organization who raises money for the Leukemia and Lymphoma Society. I have pledged to raise $950 that will go to research to help find a cure blood cancers. Please show your support by leaving a <a href="http://pages.teamintraining.org/nc/Oaks11/rrowe3" target="_blank">donation here</a>. Any amount is greatly appreciated! Thank you for your support.</p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com2tag:blogger.com,1999:blog-724993853864308794.post-55103697897058832192011-05-23T08:27:00.000-04:002011-05-23T08:27:16.351-04:00Creating a Hugo Awards DB with Clojure and Sqlite<p>In my <a href="http://rob-rowe.blogspot.com/2011/05/parsing-web-pages-with-clojure-and.html" target="_blank">previous post</a> I wrote code to retrieve the Hugo Award – Best Novel winners and nominees for the 2000s and write the results into a text file. While I was working on that post I thought it would be nice to have a database with this information. In this post I will walk you through the code I wrote to create a sqlite database using clojure.contrib.sql library.</p>
<h4>The Goal</h4>
<p>By the end of the post I will have code that will retrieve the nominees and winners since 2000 and create then load sqlite database with the parsed data . The nominees table will have the following columns: id, year, title, author, winner, read_it, own_it, want_it.</p>
<h4>The Setup</h4>
<p>The first thing to do is get sqlite installed on your machine. Since I’m working on I windows I grabbed these two downloads: <a href="http://www.sqlite.org/sqlite-shell-win32-x86-3070600.zip" target="_blank">sqlite shell</a> and <a href="http://www.sqlite.org/sqlite-dll-win32-x86-3070600.zip" target="_blank">sqlite-dll</a>. After downloading the files make sure they are in your PATH. </p>
<p>Next, I created a <a href="https://github.com/rippinrobr/hugo/tree/hugoDB" target="_blank">hugoDB</a> branch to my hugo project on github to keep the code from this post separate from the original hugo post.</p>
<p>You will also need <a href="https://github.com/technomancy/leiningen" target="_blank">leiningen</a> installed. Leiningen is a Clojure build tool that helps you manage your projects. I use in all of my clojure projects. The install takes no time at all, just follow the instructions on the <a href="https://github.com/technomancy/leiningen">project’s page</a> and you will be ready for business.</p>
<h4>Creating/Updating the HugoDB Project.clj File</h4>
<p>Once the hugoDB branch was created I updated the project.clj file to include the sqlitejdbc library. The library allows me to connect to a sqlite database. Here is the updated project file:</p>
<script src="https://gist.github.com/972515.js?file=gistfile1.clj"></script>
<p>If you want to start a fresh project, check out <a href="http://rob-rowe.blogspot.com/2011/05/parsing-web-pages-with-clojure-and.html#createproject" target="_blank">Create the Hugo Project</a> in my previous post. After creating the project add the sqlitejdbc dependency. To ensure you have all necessary dependencies run the following command:</p>
<pre>lein deps</pre>
<p>The deps command will download and install any dependencies that are not already in the project’s lib directory. Now that the project.clj file has been updated and the dependencies are in place, lets move on to the cmdline.clj file.</p>
<h4>The Code</h4>
<p>The code consists of an updated cmdline.clj file to support the database generation process and a command line option. I have also updated the hugo/parser.clj removing unnecessary functions and added a new one.  The database related code is in two files.  The first file is hugo/db/createsqlite.clj which handles the database creation and loading.  The second file, hugo/db/sqlite.clj, contains code that retrieves and inserts data. I removed the hugo/text-formatting.clj file from this branch since I am not writing out to a text file.</p>
<h5>The cmdline.clj file</h5>
<script src="https://gist.github.com/973305.js?file=cmdline.clj"></script>
<p>The –main function was updated to support command line options using the clojure.contrib.command-line/with-command-line macro. The macro makes it possible to map a command line option to a local variable. For this ‘app’ the only option is the –-drop option. When I run (-main “--drop” “true”) the local variable will contain the string “true”. The macro also has the –-help option built in. It prints the comment that is directly under the with-command-line line and a description of each option you’ve defined. Here’s what (-main “--help”) prints out when I run it in the REPL:</p>
<p><a href="http://lh5.ggpht.com/_rC3qLzTABK0/TdWpBDQTK7I/AAAAAAAAAkE/HXHd-Nr5wEk/s1600-h/image%5B5%5D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/_rC3qLzTABK0/TdWpCEqie5I/AAAAAAAAAkI/CwAAY7yff-I/image_thumb%5B3%5D.png?imgmax=800" width="572" height="141" /></a></p>
<p>After the command line support is in place I need to check to see if –-drop true. The if function is used to see if I need to drop the nominees table using the hugo.db.createsqlite/drop-table function first.</p>
<p>Next, I grab the links from the <a href="http://www.thehugoawards.org/hugo-history/" target="_blank">Hugo Awards History page</a>. Since I’m only interested in the winners from 2000 on I only pass the first 12 links to the create-and-load-db function. That is the function that kicks off the database creation and loading process begins.</p>
<p>The create-and-load-db function calls a wrapper function called get-data. I created the function so I can make use of the map function to retrieve the nominees data. The hugo.parser/parse-best-novel-nominees is called to parse the data from the web pages. I will get into the details parse-best-novel-nominees a little later. After all the data has been parsed I create the database by calling hugo.db.createsqlite/create-db function. The create-db function creates the nominees table in the hugo database. The last line of the create-and-load-db function is where the data is inserted into the newly created nominees table by calling the hugo.db.createsqlite/process-awards function.</p>
<p>That’s it for the cmdline.clj file. This file exists simply to allow me to create the database from the command line. Now lets take a look at the database code by walking you through the hugo.db.createsqlite.clj file.</p>
<h5>The hugo.db.createsqlite.clj file</h5>
<script src="https://gist.github.com/975459.js?file=gistfile1.clj"></script>
<p>The first three lines are the typical namespace and dependency declarations. I have included dependencies on my hugo.db.sqlite.clj file and the clojure.contrib.sql library giving it a short name of sql. The short name allows me to call functions in the library by prefacing them with sql/. The sql library allows me to use JDBC to access the database.</p>
<script src="https://gist.github.com/975540.js?file=gistfile1.clj"></script>
<p>hugo.db.sqlite/db contains the information needed by JDBC to locate and connect to the database. In order to create the database I needed to add the :create flag to the connection. I was able to do this by using the merge function with the db variable and :create flag allowed me to create the new-db-conn. Now I have a connection that allows me to create the database.</p>
<p>Once I had the new-db-conn connection it was time to get down to business. The ‘gateway’ function into the database creation process is the create-db function.</p>
<script src="https://gist.github.com/975597.js?file=gistfile1.clj"></script>
<p>There really isn’t much to this function but there are a few things I would like to point out. First the with-connection function will ‘wrap’ the code that makes up the body of the call which in our case the line:</p>
<pre>(create-tables)</pre>
<p>Wrap? What I mean by wrap is any database related code within the ‘with-connection’ body will use the database connection created by the with-connection call. When the body has finished the connection is closed.</p>
<script src="https://gist.github.com/975642.js?file=gistfile1.clj"></script>
<p>The create-db function contains the call to the create-tables call which is where my nominees table is created. The first parameter of the create-table function is the name of the table to create. The vectors that follow the table name are the definition for each column. Each vector has the name of the column followed by its data type. Any special description of the column like primary keys, unique, etc.. is listed after the data type.</p>
<script src="https://gist.github.com/975653.js?file=gistfile1.clj"></script>
<p>Once the database table has been created it's time to parse and load the data into the nominees table. The last call in cmdline/create-and-load-db is to the process-awards function. The process-awards function takes the parsed award data and feeds each nominee to the add-new-nominees function using the map function. The add-new-nominees function doesn’t have much meat to it. I'm finding that with Clojure you can get a lot done with very little code. First the function grabs the year from the category struct which is the first parameter to the add-nominee function. Next, the map function is used to insert a record for each entry in the category struct’s book sequence. Each item is the second parameter for the add-nominee function. When the map call completes there will be a record for each of the year's nominees in the nominees table. When the process-awards function finishes I will have a nominees table loaded with all nominees/winners since 2000.</p>
<h5>Running the HugoDB code</h5>
<p>To create the database you can either run the code from within the REPL or the command line. Here's how to run it in the REPL. First run the following command from the project’s home directory:</p>
<pre>lein repl</pre>
<p>The first time you run the app you'll just need to call the -main function like so:</p>
<pre>(-main)</pre>
<p>After you run it one time you'll need to run it with the --drop true to drop the nominees table before you start creating the database. Here's how to call the main with --drop true</p>
<pre>(-main "--drop" "true")</pre>
<p>If you choose to run it from the command line run it like this the first time: </p>
<pre>lein run</pre>
<p>After running the app once you'll need to run it like this: </p>
<pre>lein run --drop true</pre>
<p>Now that we have a database of the nominees its time to do some querying to ensure that we have loaded the data correctly. There are two ways for us to accomplish this. First I will use the sqlite3 command line tools to run SQL against the database. After that I’ll use Clojure in the REPL to show show a few select functions. Now it’s time to fire up sqlite.</p>
<h5>The Sqlite Shell</h5>
<p>Jump back to the command prompt and cd into the project’s home page. From there change into the db directory and get a listing of its contents. The directory should have a file named hugo.sqlite3.  That is the database file the app just created. To get to a querying interface run the command (assuming you have added sqlite3 to your PATH):</p>
<pre>sqlite3 hugo.sqlite3</pre>
<p>To view the tables in our database enter .tables from the sqlite prompt and you should see our single table nominees listed.& Lets make sure that the 2004 records were loaded correctly.</p>
<p><a href="http://lh5.ggpht.com/_rC3qLzTABK0/TdWpCpbVeDI/AAAAAAAAAkM/vbFhZPdvaU0/s1600-h/image%5B17%5D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/_rC3qLzTABK0/TdWpDHukI4I/AAAAAAAAAkQ/Ls8gpKUcgQk/image_thumb%5B11%5D.png?imgmax=800" width="479" height="117" /></a></p>
<p>The columns we care about here are id, year, title, author and whether or not the book was the winner plus three other columns that are there for my next post. The first record has 1 in the winner column which indicates it was the winner. All of the nominees for the year were also properly saved.</p>
<h5>The hugo.db.sqlite.clj file</h5>
<p>In addition to the sqlite shell I wrote a few functions that will retrieve the nominees from the database. The first one I will call use is get-nominees. It does what you might expect returns all of the records in the nominees table.</p>
<script src="https://gist.github.com/977476.js?file=gistfile1.clj"></script>
<p>The get-nominees function introduces function overloading in Clojure. If get-nominees is called without parameters it will call get-sql passing in the sql statement defined in the get-all-nominees var which returns all of the records. However, if a year is passed in the function will return all winners/nominees for the given year. Before calling the get-sql function I add a predicate to base sql statement with a place holder. The new string is the first item in the vector. The next item is the value that will replace the placeholder when get-sql calls the with-query-results function. Really not much to get-nominees, most of the work is done in the get-sql function.</p>
<p>The functions that all the ‘get’ type functions are based on the function sqlite.clj/get-sql . It wraps the call to clojure.contrib.sql/with-query-results function. The first parameter is a sequence that will contain the results of the query. The next parameter is the sql statement and parameters to be ran. The doall statement forces the lazy sequence that contains the results into a ‘real’ sequence that is returned to the caller.</p>
<h4>Summary</h4>
<p>Creating a sqlite database and running queries against it with Clojure is straight forward. Adding support for command line options is trivial. In my next 'Hugo' post I am going create a UI that will allow me to view the data. Stay tuned!</p>
<p>As part of my Clojure learning process I appreciate any and all comments on my code. Following my last post I had great comments that helped improve my code and expand my Clojure knowledge. Please keep the comments coming.</p>
<h4>Resources</h4>
<p><a href="http://clojure.org/" target="_blank">clojure</a>, <a href="http://code.google.com/p/clojure-contrib/" target="_blank">clojure-contrib</a>, <a href="https://github.com/cgrand/enlive" target="_blank">enlive</a>, <a href="https://github.com/technomancy/leiningen" target="_blank">leiningen</a>, <a href="http://www.sqlite.org/" target="_blank">sqlite</a>, <a href="http://rob-rowe.blogspot.com/2011/05/parsing-web-pages-with-clojure-and.html" target="_blank">my previous post</a>.</p>
<h4>Code</h4>
<p>The code for this project can be found on the <a href="https://github.com/rippinrobr/hugo/tree/hugoDB" target="_blank">hugoDB branch of the Hugo project</a>.  You can download the entire hugoDB branch of the project <a href="https://github.com/rippinrobr/hugo/archives/hugoDB" target="_blank">here</a>.  The code and database files I discussed in the post can be viewed here:  <a href="https://github.com/rippinrobr/hugo/blob/hugoDB/src/cmdline.clj" target="_blank">cmdline.clj</a>, <a href="https://github.com/rippinrobr/hugo/blob/hugoDB/src/hugo/db/createsqlite.clj" target="_blank">createsqlite.clj</a>, <a href="https://github.com/rippinrobr/hugo/blob/hugoDB/src/hugo/db/sqlite.clj" target="_blank">sqlite.clj</a> and <a href="https://github.com/rippinrobr/hugo/tree/hugoDB/db" target="_blank">the database</a>.</p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com3tag:blogger.com,1999:blog-724993853864308794.post-30333260526067606522011-05-10T16:01:00.000-04:002011-05-14T15:04:00.237-04:00Parsing Web Pages with Clojure and enlive<p>
As my infatuation with Clojure grows I thought I would write some code to retrieve all of the works that have either won or been nominated for the <a href="http://www.thehugoawards.org" target="_blank">Hugo award's Best Novel category</a>. I know it’s geeky but it is information that I can use so why not use it as a source to learn more Clojure?</p>
<p>
Please keep in mind that I am writing this blog from my perspective as a Clojure noob. Any and all feedback on the post or the code is welcome.  Even if you think it is minor, please pass it along. With that said, lets get on with post!</p>
<h4>
Goal</h4>
<p>
By the end of the post is to have code that will retrieve the winners and nominees for the Best Novel category since 2000 and write them to a text file with the following layout:</p>
<pre>Year Hugo Awards - Best Novel
Title – Author (Winner)
Title – Author
Title – Author ...</pre>
<h4>
The Setup</h4>
<p><em><b>UPDATE</b> As @Bendlas mentioned below leiningen will install the clojure jars. You can skip the first paragraph and go right to installing leiningen. I have tested it on a Ubuntu VM and when I ran lein repl leiningen downloaded the clojure jars. On Windows you will need to have curl.exe or wget.exe installed to get it to work. Thanks again @Bendlas.</em></p>
<p style="text-decoration: line-through;">
If you don’t already have Clojure installed you can get everything you need from the <a href="http://clojure.org/downloads">download page</a>. While you are there go ahead and grab the clojure-contrib.zip as well. Assuming you already have Java on your machine the next step is to add the Clojure and clojure-contrib directories to your CLASSPATH.</p>
<p>
Once you have Clojure installed the next thing to install is <a href="https://github.com/technomancy/leiningen">lein</a>. It is a Clojure build tool that helps you manage your projects. In my short period of time in the Clojure world lein has been a great tool and I have found it has many useful plugins. The install takes no time at all, just follow the instructions on the <a href="https://github.com/technomancy/leiningen">project’s page</a> and you will be ready for business. Now that Clojure and lein are installed I'm ready to start the ‘Hugo’ project.</p>
<a name="createproject"></a>
<h3>
Creating the Hugo Project</h3>
<p>
To create the project using run lein with the parameters below:</p>
<pre>lein new hugo</pre>
<p>
The command will create a directory structure for our project. For a little more information on the project directory structure lein creates take a look at my previous post <a href="http://rob-rowe.blogspot.com/2011/03/getting-started-with-ring-and-compojure.html" target="_blank">Getting Started with Ring and Compojure - Clojure Web Programming</a>.  I have a little more detail there.</p>
<p>
I updated the project.clj file to include dependencies on enlive and clojure-contrib. A line was added to indicate which namespace my main function is located in. The new line allows me to run this 'app' from the command line.</p>
<script src="https://gist.github.com/934268.js?file=project.clj"></script>
<p>
The project setup is complete, now its time to start parsing!</p>
<h3>
The Code </h3>
<p>
The code for this project is in three source files under the project’s src directory.  The cmdline.clj file houses the app’s main function which allows the app to be started from the command line. The hugo/parser.clj file contains the code that retrieves and parses the web pages.  The last file is hugo/text-formatting.clj which contains the code to format the output.  In this post I will walk through the cmdline.clj and hugo/parser.clj files.</p>
<h4>
The cmdline.clj file</h4>
<script src="https://gist.github.com/934721.js?file=cmdline.clj"></script>
<p>
As you can see, there isn’t much to this file.  The file exists to create a class that allows me to run hugo from the command line and provides a concise way to run the code within the REPL. The first five lines set up the namespace, include my code’s namespaces and loads the clojure-contrib.duck-streams library which I will use to write the results out to the output file.  </p>
<p>
Since I want to run this application from the command line I need to generate a java class file.  To do this I use the :gen-class macro.  The generated cmdline.class file will be placed in the project’s classes directory. Any methods in the java class will look in the source clj file for a method by the same name but preceded by a –.  That is why the –main function exists.  It is what is called when I execute the app outside of the REPL.</p>
<p>
The –main method calls the hugo.parser/get-award-links function to retrieve the links to each year’s awards page from the Hugo awards history page.  The links are returned as a sequence, since I only want the entries for 2000 to 2010 the code grabs the first 12 links, which are passed to the prep-for-file function.  </p>
<p>
The prep-for-file function is where the real parsing is kicked off, I will discuss the parsing in more detail later. For now just know that the data retrieved from the URL is formatted by the hugo.text-formatting/format-output function and the map function. The results are converted to a string by using the apply and str functions. </p>
<p>
When the parsing and formatting is complete the results are passed to the clojure.contrib.duck-streams/spit function. The spit function, I really like that name, writes the results to a file named hugo_awards_best_novel.txt. That's it. I've given you the 5 second tour of the cmdline.clj file, now its time to take a look at the HTML parsing.</p>
<h4>
The hugo/parser.clj file</h4>
<p>
As you might expect this is where all the parsing code lives. The first function called is the get-award-links. The function is responsible for parsing out all of the links to the annual awards pages.</p>
<h5>
The get-award-links Function</h5>
<script src="https://gist.github.com/936586.js?file=gistfile1.clj"></script>
<p>
The first task this function does is to retrieve the page’s HTML tags using the fetch-url function which wraps enlive’s html-resource function. The html-resource function retrieves a web page and returns its HTML tags in a sequence that is passed to other enlive functions as input. </p>
<p>Once the page has been parsed, I call enlive’s select function passing the tag sequence as the first parameter. Select's second parameter tells the function which tags I want out of the tags sequence using something similar to CSS selectors. In this case I’m telling select to grab all a tags inside of LI tags that are members of the page_item class and are within DIV tag that with the id of content. The second vector tells select that I want the text for each link so I can use it for the year value later. After the parsing of the tags is done the map function will grab the attrs for each tag that is returned which returns the following for each link:</p>
<p>
<a href="http://lh4.ggpht.com/_rC3qLzTABK0/TckntUg152I/AAAAAAAAAj0/Sbx1rZnUwt8/s1600-h/image4.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/_rC3qLzTABK0/TcknthiX4JI/AAAAAAAAAj4/mwYjI9_YaYI/image_thumb2.png?imgmax=800" width="598" height="71" /></a></p>
<p>
When this function call is completed a map is returned with a title and href for each year that the Hugo awards were given.  The results of this call are passed to the prep-for-file function in the cmdline.clj file.  </p>
<h5>
The get-awards-per-year Function</h5>
<script src="https://gist.github.com/962679.js?file=gistfile1.clj"></script>
<p>
Now that I have the links to all the award pages it is time to gather all the data on each category. The function creates a sequence of category structs that contain the award category, the nominees/winners in the category and the year the award was given. </p>
<p>The year’s page is retrieved using the fetch-url function and the results are stored in the page-content variable. Next, the parse-award-page function is called passing in the page-content as its only parameter. It returns a sequence that contains lists for each award given that year that will look like this: ((“Best Novel”) (array maps for each nominee/winner)). I will refer to this sequence as the category sequence from here out. Right now the parse-award-page function looks like a black box I promise to get into the details in a bit.</p>
<p>
The results of the parse-award-page call are passed map to create a sequence that contains category structs.  The category struct is defined as:</p>
<script src="https://gist.github.com/964420.js?file=gistfile1.clj"></script>
<h5><em>Getting the award string</em></h5>
<p>
In the map function call I am using an anonymous function to create a category struct. When the map call completes it returns a sequence of category structs for the given year. Creating a new struct is easy, just pass in the name of the struct to create and a value for each of the keys in the struct. The category struct's first key is the :award key. The value is parsed with this code: (apply str (first %)). Since the first item in the category sequence is a string in a lazy sequence representing the award's title I need to use apply str instead of just str. If I called (str (first %)) what I would get back is something like this: clojure.lang.LazySeq@5784711f which is obviously not what we want.  </p>
<h5>
<em>Getting the books sequence</em></h5>
<p>
Grabbing the books that represent the nominees/winners is almost as easy as the award. Since I know that the nominee/winners are stored in the second part of the category sequence lists I use the second function to retrieve them.</p>
<pre>(get-book-info (rest (second %)))</pre>
<p>
I’m using rest here because for some reason the first entry in the sequence is “\n” I’m not sure why. In the future I will figure it out but for now I’m using the rest call to get to the ‘guts’ of the book sequence. The results of the rest call are passed to a helper function that returns a sequence of work structs that will be stored in the category struct’s books key.</p>
<h5>
<em>Getting the year string</em></h5>
<p>
The last key in the category struct is the year key. It will store a string that begins with the year and ends with “Hugo Awards”.  The code to retrieve the ‘year’ makes use of the select statement, grabs the first element in the returned sequence and converts the value of :content to a string. Here's what the code looks like: </p>
<p>
(apply str (:content (first (html/select page-content #{[:div#content :h2]}))))</p>
<p>
Now I have a value for each of the category struct’s keys. The struct provides a much easier way to work with the data. At this point all of the parsing has been completed. All that is left is for the cmdline/prep-for-file function to format the data and write it out to the file. Since that is pretty straight forward I'm going to leave that code out of the post. Before I wrap up this post I’d like to dive into the hugo.parser/parse-award-page function, where the real parsing happens.</p>
<h5>
The parse-award-page Function</h5>
<script src="https://gist.github.com/963002.js?file=gistfile1.clj"></script>
<p>
Once the year’s award page has been retrieved, its tag sequence function is passed to parse-award-page. The function grabs the category title and the nominees/winners and creates a sequence of lists. Here’s how it is done. All of the nominees/winners are found in the map function call. The sequence returned from the call to select returns all UL tags found in content DIV tag. Each tag is passed to the anonymous function which just pulls out the :content key from the tag’s array map creating a sequence of book titles.</p>
<p>
The category titles are parsed on the line that has the split-at function call. Again, the select function is called to find all P tags that are within the content DIV. The text for the the first child of the P tag is returned creating a sequence of category titles. The split-at function is called to ‘remove’ the first four P tag results since the contain information on where the awards banquet was held.</p>
<p>
After both the titles and then nominees/winners sequences are created the interleave function is called. Interleave creates a single sequence by combining the two sequences one item at a time. How the function works is the first item in the titles sequence is added to the new sequence followed by the first item in the nominees sequence, the second from titles is followed by the second nominees item, etc. When interleave returns I have one sequence that looks something like ( “award title” “nominees” “award title” “nominees”….).</p>
<p>
Having the sequence provided by interleave is nice but it isn’t going to work for what I want. I need to pair the category title with the nominees/winners for the category. This is where the partition function comes in. According to the <a href="http://clojure.github.com/clojure/clojure.core-api.html#clojure.core/partition" target="_blank">partition documentation</a> the function will “create a lazy sequence of n items” which in our cause is 2. When the parse-award-page function completes it returns a sequence of lists that match the category up with it’s nominees/winner which is exactly what I need in the get-awards-per-year function. </p>
<h4>How do I run it from the command line?</h4>
<p>
If you are like me most of the clojure you write is either run through the REPL or as a web app. I had no idea how to run this ‘app’ from the command line. After checking out the <a href="https://github.com/technomancy/leiningen" target="_blank">leiningen project</a> again I noticed that there is a command called uberjar. What uberjar does is create a jar file that bundles everything up that is needed to run your app from the command line. The jar file uses the naming convention of:</p>
<p>
<project name>-<version info>-standalone.jar</p>
<p>
<em>Remember I’m a .NET guy by day so I don’t have a real in-depth knowledge of jar files yet. I just know that they allow my to run the app from the command line.</em> Once the jar file has been created I can run the app from the command line using this command: </p>
<pre>java -jar hugo-0.0.3-SNAPSHOT-standalone.jar </pre>
<h4>
Summary</h4>
<p>
Parsing HTML using clojure is relatively easy using <a href="https://github.com/cgrand/enlive" target="_blank">enlive</a>. Using enlive I was able to parse the Hugo Awards information to create a text file with all of the Best Novel category nominees and winners ( <a href="https://github.com/rippinrobr/hugo/raw/master/hugo_awards_best_novels.txt" target="_blank">hugo_awards_best_novels.txt</a> ) since 2000. </p>
<h4>
One More Thing…</h4>
<p>
When you run the project you may encounter an IOException like this:</p>
<p>
<a href="http://lh3.ggpht.com/_rC3qLzTABK0/TclWWppfZEI/AAAAAAAAAj8/5F9WSeSJbUw/s1600-h/image%5B4%5D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/_rC3qLzTABK0/TclWWwbvR_I/AAAAAAAAAkA/mQLpRDh56XU/image_thumb%5B1%5D.png?imgmax=800" width="611" height="52" /></a></p>
<p>
You can resolve the issue by visiting the URL through a web browser. I believe I can get around this issue by setting the user-agent for my enlive html-resource call but I couldn’t figure out how to do it. If anyone has a suggestion please leave me a comment.</p>
<h4>
Resources</h4>
<p>
<a href="http://clojure.org/" target="_blank">clojure</a>, <a href="https://github.com/richhickey/clojure-contrib" target="_blank">clojure-contrib</a>, <a href="https://github.com/cgrand/enlive" target="_blank">enlive</a>, <a href="https://github.com/technomancy/leiningen" target="_blank">lein</a></p>
<h4>
Code</h4>
<p>
<a href="https://github.com/rippinrobr/hugo" target="_blank">Download entire project</a>. Code files: <a href="https://github.com/rippinrobr/hugo/blob/master/src/cmdline.clj" target="_blank">cmdline.clj</a>, <a href="https://github.com/rippinrobr/hugo/blob/master/src/hugo/parser.clj" target="_blank">parser.clj</a> and <a href="https://github.com/rippinrobr/hugo/blob/master/src/hugo/text_formatting.clj" target="_blank">text_formatting.clj</a>.  </p>
<p>
The output file: <a href="https://github.com/rippinrobr/hugo/raw/master/hugo_awards_best_novels.txt" target="_blank">hugo_awards_best_novels.txt</a></p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com11tag:blogger.com,1999:blog-724993853864308794.post-23002334883958948272011-03-28T19:43:00.000-04:002011-05-27T14:57:22.476-04:00Getting Started with Ring and Compojure - Clojure Web Programming<p>I am currently testing out different technologies to run a small web service in a windows environment at my day job. I have worked with Node.js on Linux but have not had much luck with it on windows. At the same time I am also learning Clojure so I did some checking to see what type of web development options are available in Clojure and I found <a href="https://github.com/mmcgrana/ring" target="_blank">Ring</a> and <a href="http://weavejester.github.com/compojure/" target="_blank">Compojure</a>. In this post I will follow the same process I did in my <a href="http://rob-rowe.blogspot.com/2011/03/getting-started-with-nodejs.html" target="_blank">Getting Started with Node.js</a> post, that is to start off with a bare bones ‘Hello World’ sample with <a href="https://github.com/mmcgrana/ring" target="_blank">Ring</a> and then follow that up with a sample using <a href="http://weavejester.github.com/compojure/" target="_blank">Compojure</a> to receive parameters by GET and POST.</p>
<h3>The Setup</h3>
<p>If you don’t already have Clojure installed you can get everything you need from the <a href="http://clojure.org/downloads" target="_blank">download page</a>. While you are there go ahead and grab the clojure-contrib zip as well. Assuming you already have Java on your machine the next step is to add the Clojure and Clojure-contrib directories to your CLASSPATH.</p>
<p>Once you have Clojure installed the next thing to install is <a href="https://github.com/technomancy/leiningen" target="_blank">lein</a>. It is a Clojure build tool that helps you manage your projects. In my short period of time in the Clojure world lein has been a great tool and I have found it has many useful plugins. The install takes no time at all, just follow the instructions on the <a href="https://github.com/technomancy/leiningen" target="_blank">project’s page</a> and you will be ready for business. Now that I have Clojure and lein installed I'm ready to start the ‘Hello Clojure Web’ project.</p>
<h3>Creating the Hello Clojure Web App</h3>
<p>Step number one is to create the project using lein by running the command:</p>
<code>lein new hello-clojure-web</code>
<p>When lein has finished there will be a new directory named hello-clojure-web. When you cd into the directory and do dir or ls you should see a directory structure similar to this:</p>
<p><a href="http://lh5.ggpht.com/_rC3qLzTABK0/TZClvf3npxI/AAAAAAAAAjU/h18FPZJVCEM/s1600-h/image%5B9%5D.png"><img style="background-image: none; border-right-width: 0px; margin: 0px auto 10px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/_rC3qLzTABK0/TZClv9CDueI/AAAAAAAAAjY/qckwnan1rL8/image_thumb%5B4%5D.png?imgmax=800" width="518" height="244" /></a></p>
<p>Lein creates a directory structure for our project separating the tests from the source code, a .gitignore file and a project.clj file. The project.clj file it handles project dependencies and sets up variables that are used to describe the project. We will just scratch the surface of the project file in this post.</p>
<script src="https://gist.github.com/887441.js?file=project.clj"></script>
<h4>The project.clj File</h4>
<p>In the project.clj file the first line simply defines the name and version of the project. The dependencies section is used to list what libraries the project depends on. For our app we need to add the dependencies to the ring-core and ring-jetty-adapter libraries. To get the exact information I needed to add in the dependencies section I turned to the <a href="http://clojars.org/ring/ring-core" target="_blank">clojars.org</a> site. The clojars site is a place where you can find open source libraries. It provides you with the exact syntax required to add the library to your project. To see what a clojars page looks like visit the <a href="http://clojars.org/ring/ring-core" target="_blank">ring/ring-core</a> page. The line below adds the two ring dependencies I need for my project. Dependencies are added by using the libraries name followed by the version you wish to use.</p>
<code>[ring/ring-core "0.3.7"][ring/ring-jetty-adapter "0.3.7"]</code>
<p>After we've added to the Ring dependency our project.clj file should look like:</p>
<script src="https://gist.github.com/887501.js?file=project.clj"></script>
<p>After adding my dependencies to the project.clj file I need to 'install' them. Lein will do it for me when I run this command:</p>
<code>lein deps</code>
<p>Lein will retrieve the necessary files and place them in the pr<a href="http://lh4.ggpht.com/_rC3qLzTABK0/TZClwVG6fgI/AAAAAAAAAjc/YhfkZa-tFVQ/s1600-h/image14.png"><img style="background-image: none; border-right-width: 0px; margin: 10px 0px 0px 10px; padding-left: 0px; padding-right: 0px; display: inline; float: right; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" align="right" src="http://lh5.ggpht.com/_rC3qLzTABK0/TZClwlBW6lI/AAAAAAAAAjg/E4U1cMy4LBU/image_thumb10.png?imgmax=800" width="423" height="61" /></a>oject's lib directory. If all goes well you should see something like the image to the right. If you see an error message text similar to this towards the bottom of the message:</p>
<pre>1 required artifact is missing.<br /><br />for artifact:<br /> org.apache.maven:super-pom:jar:2.0</pre>
<p>It usually means that you have misspelled the name of a dependency in the project.clj file. To see it for yourself remove a letter from the dependency we just added, save the file and then re-run the lein deps command.</p>
<h4>src/hello-clojure-web/core.clj</h4>
<p>When I created the project lein created a core.clj file in the src/hello-clojure-web directory. The only line inside of the file is the project's namespace declaration. My first step was to add a reference to the ring.adapter.jetty library which I will us to run our HTTP service. Next, I added the handler function which handles the web requests. The last line of the file starts the web service passing the handler function to the run-jetty function. Here is the completed hello-clojure-web/core.clj file.</p>
<script src="https://gist.github.com/887596.js?file=core.clj"></script>
<p>The handler function takes one parameter which is the web request and returns the response. It hanldes all web requests that come on on port 8080 returning the same response for a request at ‘/’ and ‘/this/is/a/long/one’.</p>
<p>In order to test our project out we will use the REPL. Starting the REPL session with lein makes it easier to run our code within the REPL session. To start a REPL session I need to jump back to the command prompt in the project's home directory and run:</p>
<code>lein repl</code>
<p>From within the REPL session enter the following line:</p>
<code>(use 'hello-clojure-web.core)</code>
<p>This will starts up my service which is now listening for requests on port 8080. Next, I fired up Chrome and entered <a href="http://localhost:8080/">http://localhost:8080</a>. If everything works correctly I should see Hello Clojure Web! If you do not see the text then an error has occurred. Usually the REPL gives an error message that will point you in the right direction.</p>
<p>I want to change the text to reflect the fact I'm using Ring. I'm going to keep the REPL running and go back to the source file. I changed the Hello Clojure Web string to read Hello Ring!, saved it and refreshed the browser but I didn't see the changes. When I stop and restart the REPL. Re-enter the use statement and reload the web page I will see my changes.</p>
<p>In order for me to see my changes without restarting the REPL I need to add a reference to the ring-devel library. Since I do not want this library to be apart of my ‘production’ version I will only add the dependency to my dev environment by making use of the dev dependency tag. My updated project.clj file now looks like this. </p>
<script src="https://gist.github.com/887646.js?file=project.clj"></script>
<p>After the changes make sure to run lein deps again. You will see in the output of that command it places files in the hello-clojure-web/libs/dev directory in addition to the hello-clojure-web/libs directory.</p>
<p>Now that I have the dependencies taken care of it is time to update the code in the core.clj file. I've wrapped the run-jetty call in a function called boot so I can make use of the wrap-reload function. What this call does is reload the namespace of our application before each request is handled. So now we can make changes to our code and refresh the browser to see them. Here's what the updated code looks like:</p>
<script src="https://gist.github.com/887845.js?file=core.clj"></script>
<p><a href="http://lh3.ggpht.com/_rC3qLzTABK0/TZClxAtW3PI/AAAAAAAAAjk/xPge9QF46fs/s1600-h/image%5B4%5D.png"><img style="background-image: none; border-right-width: 0px; margin: 10px 10px 10px 0px; padding-left: 0px; padding-right: 0px; display: inline; float: left; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" align="left" src="http://lh6.ggpht.com/_rC3qLzTABK0/TZClxY9sRgI/AAAAAAAAAjo/qRZ7Yoowt_E/image_thumb%5B1%5D.png?imgmax=800" width="324" height="149" /></a>As you can see there wasn't much change in the code needed to be able to handle our updates. To see it in action, start up REPL and view the app in the browser. Change the response text and then reload the web page. You should see our updated message. Next I'll describe how to parse parameters from a GET and POST requests.</p>
<h3>Retrieving Parameters</h3>
<p>In this section I'm going to introduce <a href="https://github.com/weavejester/compojure" target="_blank">Compojure</a>, a small, open source web framework. To keep things simple I am going to add the Compojure code to our already existing core.clj file. Before I make changes to core.clj I need to update the project.clj file with the Compojure dependencies.</p>
<script src="https://gist.github.com/889293.js?file=project.clj"></script>
<h4>The Updated project.clj file</h4>
<p>I added the compojure lib in the 'normal' dependencies section and in the dev-dependencies I added the reference to the lein-ring plugin. The ring plugin allows me to start ring by running the command: lein ring server. For the ring server command to work I need to tell ring where the handler method is which is done on the last line of the file. Ring knows that app will handle the web requests. I ran lein deps again to update the project's dependencies to include lein-ring.</p>
<p>Now that the project.clj file is updated I need to adjust the ns statement in the core.clj file to include the Compojure libraries:</p>
<pre>(ns hello-clojure-web.core <br /> (:use compojure.core) <br /> (:require [compojure.route :as route] <br /> [compojure.handler :as handler]) <br /> (:use ring.middleware.reload) <br /> (:use ring.adapter.jetty)) </pre>
<p>Nothing special going on here, just a few more libraries to include. Next I setup my routes. Using the defroutes macro I create the routes I want my app to respond to. Compojure will take the route definitions and generate a Ring handler function. The routes are processed from the top down, the first match wins. If a request comes in that doesn’t match the three routes I’ve defined the route/not-found will be called. </p>
<script src="https://gist.github.com/889545.js?file=gistfile1.clj"></script>
<p>My last step is to bind app to the handler/site my-routes function. This binding is used to start up the application. After I have updated the file I can start the app by running:</p>
<code>lein ring server</code>
<p>The command starts up the web app and brings up a browser opening it to http://localhost:3000/. To test my app I wrote a batch script that utilizes curl. It tries my three routes plus a non-existing route to ensure they are handled properly, and it worked!</p>
<p><a href="http://lh4.ggpht.com/_rC3qLzTABK0/TZClxvsVATI/AAAAAAAAAjs/IdEzAZTMm9M/s1600-h/image%5B8%5D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/_rC3qLzTABK0/TZClyOJZDWI/AAAAAAAAAjw/djtWgpg1ad0/image_thumb%5B3%5D.png?imgmax=800" width="620" height="120" /></a></p>
<p>The complete core.clj file:</p>
<script src="https://gist.github.com/889572.js?file=core.clj"></script>
<h3>Summary</h3>
<p>The Clojure web world offers frameworks at different levels. You can stay at a relatively low level buy using Ring or if you want something at a little higher level you can use Compojure. I am a Clojure noob and I was able to get a web app up and running in no time. I will be expanding on my Compojure knowledge through a project I am working on with my son. We had originally planned on using Node.js but now that I found Compojure we have decided to go with a Clojure based application. As the project progresses I am sure I will have more Clojure web related posts.</p>
<h3>Resources</h3>
<p><a href="http://clojure.org/" target="_blank">Clojure</a>, <a href="https://github.com/technomancy/leiningen" target="_blank">Lein</a>, <a href="https://github.com/mmcgrana/ring" target="_blank">Ring</a>, <a href="https://github.com/weavejester/compojure" target="_blank">Compojure</a></p>
<h3>Code</h3>
<a href="https://gist.github.com/890509" target="_blank">project.clj</a> <a href="https://gist.github.com/890500" target="_blank">core.clj</a> <a href="https://gist.github.com/891526" target="_blank">test_clojure_web.bat</a>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com9tag:blogger.com,1999:blog-724993853864308794.post-68208506231787439052011-03-18T14:16:00.001-04:002011-03-18T14:23:14.465-04:00Getting vsClojure up and running in VS2010<p>This is a quick post on how to get the Visual Studio 2010 vsClojure Extension manager up and running.</p> <h3>Step 0. Install The Extension</h3> <p>Fire up Visual Studio and go to the Tools menu. From there select the the ‘Extension Manager..’ option. When the Extension Manager click on the ‘Online Gallery’ bar on the left hand side of the window. Next enter vsClojure on the upper right corner of the window. Your window should look, something like this:</p> <p><a href="http://lh5.ggpht.com/_rC3qLzTABK0/TYOhYH8J1zI/AAAAAAAAAi0/KV68Myop_c8/s1600-h/image%5B4%5D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/_rC3qLzTABK0/TYOhYbGp1EI/AAAAAAAAAi4/2z7an5inMHs/image_thumb%5B2%5D.png?imgmax=800" width="596" height="159" /></a></p> <p>Now click on the download button and follow the directions.  When the extension has been installed you will need to restart visual studio if it doesn’t do it for you. </p> <h3>Step 1. Create a Clojure Project</h3> <p>When Visual Studio has restarted close any solutions that you my have open and create a new Clojure project. When you do a File|New or Click on the New Project link from the startup page you should see Clojure listed under the ‘Installed Templates’ section on the left:</p> <p><a href="http://lh5.ggpht.com/_rC3qLzTABK0/TYOhZ3D69UI/AAAAAAAAAi8/recyZy-miFs/s1600-h/image%5B8%5D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/_rC3qLzTABK0/TYOhak5TxlI/AAAAAAAAAjA/eT4muyEsOj8/image_thumb%5B4%5D.png?imgmax=800" width="608" height="168" /></a></p> <p>Select Clojure, name the project whatever you want and click the OK button. When the project is created you will have a project with a file named program.clj and a reference to the Clojure.dll file.</p> <p><a href="http://lh4.ggpht.com/_rC3qLzTABK0/TYOha_6hKXI/AAAAAAAAAjE/FyxYj9eUuPc/s1600-h/image%5B15%5D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/_rC3qLzTABK0/TYOhbC_T31I/AAAAAAAAAjI/XBRBmXfC0Mo/image_thumb%5B9%5D.png?imgmax=800" width="522" height="281" /></a></p> <p>The first time I installed the extension and created a project the reference to the Clojure.dll was to a dll in a location that did not exist, “\1.2.0\Clojure.dll”. I did a quick search and  <a href="https://groups.google.com/group/vsclojure-extension/browse_thread/thread/73f203c1b5645fec" target="_blank">found this post</a> and it resolved my issue.  I only had this problem the first time I installed the extension all subsequent installs have worked flawlessly.</p> <h3>Step 2 – Run the Project</h3> <p>Running a Clojure project is done just like running a C# project, by using your f5 key. When I do that I see:</p> <p><a href="http://lh6.ggpht.com/_rC3qLzTABK0/TYOhbgSEGlI/AAAAAAAAAjM/XqvsKYrwE5g/s1600-h/image%5B19%5D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/_rC3qLzTABK0/TYOhb3-e7gI/AAAAAAAAAjQ/OtX_wt6Zgdo/image_thumb%5B11%5D.png?imgmax=800" width="586" height="145" /></a></p> <p>If you receive an error when you try and run the project that starts with:</p> <pre>\1.2.0\Clojure.Compile core The system cannot find the path specified. </pre><br /><br /><p>Follow <a href="https://groups.google.com/group/vsclojure-extension/browse_thread/thread/73f203c1b5645fec" target="_blank">these instructions</a> and it should resolve your problem.</p><br /><h3>Summary</h3><br /><p>Not an earth shattering blog post by any means but if you are a .NET person looking to expand his/her horizons like myself this may be a way to get you exposed to Clojure While I go through my Clojure book(s) I will be working with both the ‘real’ Clojure (JVM based) and Clojure CLR (since I work in a .NET environment).</p><br /><br /><p><a href="http://clojure.org/" target="_blank">Clojure Site</a>, <a href="https://github.com/richhickey/clojure-clr/wiki/" target="_blank">Clojure CLR site</a> and the <a href="https://github.com/jmis/vsClojure" target="_blank">VS Extension’s site</a></p><br /><br /><p>I have a feeling there will be more Clojure posts in the not too distant future.</p> Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com2tag:blogger.com,1999:blog-724993853864308794.post-58902167411208011942011-03-08T11:32:00.000-05:002011-03-16T14:45:14.153-04:00Getting Started with Node.js<a href="http://nodejs.org/" target="_blank">Node.js</a> is a server-side, event-driven environment that I am using to wrap my code generation scripts up and present them as a service. I wanted a HTTP service that was simple and quick without much ‘fluff’. Node.js definitely comes without ‘fluff. In this post I’m going to give you a quick overview of Node. It is not a deep dive but a quick example of ‘normal’ web requests.<br />
<h3>The Setup</h3>
One thing I need to point out that Node.js is primarily a linux/unix/bsd solution. Supposedly you can run it on windows but it is not recommended. In my development environment I am running it on a Ubuntu 10.04 VM. I installed node and its package management system (npm) by following these <a href="https://github.com/joyent/node/wiki/Installation" target="_blank">instructions</a>. Since the instructions are clear I won’t duplicate them here. After node is installed it is time for a quick test.<br />
<h3>Hello World Node.js Style</h3>
Actually I didn’t write my own ‘Hello World’ I just copied the one from the node.js home page. Here’s what it looks like:<br />
<script src="https://gist.github.com/859451.js?file=hello-world.js">
</script> <br />
The code is pretty straight forward. The first line creates a the http object that we’ll use to create our HTTP server. Next, the script creates a web server object that takes in a function that is added to the request event queue. In the body of the function the first line creates the response’s header. The second line sends our Hello, World statement back to the browser. Lastly, the http server object’s listen method is called to tell the server to listen on the port 8124 on the local address.<br />
To start the server I simply type: <br />
<pre>node hello-world.js</pre>
Once the server has started the console.log message is printed which shows that the service is open for business. The hello world script works but it will send the hello world message to any GET request it receives on the port, http://localhost:8124 , http://localhost:8124/rob/was/here, etc all return the same message. You can set up 'straight' node.js to respond to different urls but I found an easier way to do it using express.js<br />
<h3>Introducing Express.js</h3>
<a href="http://expressjs.com/" target="_blank">Express.js</a> is a package that will turn our node.js script into something that looks like the JavaScript version of Sinatra. Before I show you what the code looks you need to install express.js.<br />
<h4>Installation</h4>
If you followed the instructions for installing node.js to the letter you’ve already installed the package manager (npm). If you skipped the npm install earlier, install npm now. To install express simple run this from the command line:<br />
<pre>npm install express</pre>
That’s it. Now we can use it to help us build out our service.<br />
<h3>Hello World – Express.js version</h3>
Here’s our hello world service with express:<br />
<script src="https://gist.github.com/859653.js?file=hello-world-express.js">
</script>
<br />
The first line is similar to the pure node.js one except that I create the app object after ‘requiring’ the library. The next line is where the express.js goodness kicks in. The line is is telling the server that whenever you receive an HTTP GET request for the url ‘/’ run the function that is passed into the app.get method call. Now, If start the service and run: <br />
<pre>curl http://localhost:8124</pre>
I will see 'Hello World' written to STDOUT. If I tack on /Rob to the end of the URL I will receive an error that states: <br />
<pre>Cannot GET /Rob</pre>
<h3>Retrieving Parameters (GET version)</h3>
So what if I want to pass the string Rob in as a parameter, how would I do that with express.js? It is pretty easy. Here is an updated version of the code that supports passing in a name as a parameter.<br />
<script src="https://gist.github.com/860262.js?file=gistfile1.js">
</script>
<br />
As you can see there isn’t much to adding parameters. A few things I’d like to point out are that the ? after the parameter name means that the parameter is not required. If one is passed it will be used. If no value is passed in we will still respond to the / request. Parameters are accessed by making referencing: <br />
<pre>req.params.<variable name here></pre>
That’s great but how do I do POST requests with express.js?<br />
<h3>Retrieving Parameters (POST version)</h3>
How do you respond to a POST request? I’m sure you’ve probably already figured that out but I’ll show you the code to handle a post:<br />
<script src="https://gist.github.com/860296.js?file=gistfile1.js">
</script>
<br />
The app.post method is almost the same as the app.get method. If you look at the parameters to the fs.writeFile method you will notice the one visible difference in this code snippet, the parameters are accessed via the req.body object. I’m going to start the service and then call it with the following command:<br />
<pre>curl -d "path=/tmp/blog.txt&content=Hi, Rob\n" http://localhost:8124/savefile</pre>
When I run that I get the following error:<br />
<script src="https://gist.github.com/860337.js?file=gistfile1.txt">
</script>
<br />
Why did I receive this error? Out of the box Express.js doesn’t know how to parse the variables from a POST. In order to access the POST variables with express we need to add a piece of middle ware one line to our script:<br />
<pre><s>app.use(express.bodyDecoder());</s></pre>
<pre>app.use(express.bodyParser());</pre>
<em><b>Updated</b> after updated Express.js - Thanks Douglas</em><br/><br/>
Now express.js will be able to parse the parameters and store them in the req.body object. If I re-start the service and run the curl statement I should see the statement ‘File Saved!’ and have a text file at /tmp/blog.txt with the content ‘Hi, Rob’. Handling POST requests are almost as easy as handling GETs.<br />
The full hello-world-express.js script:<br />
<script src="https://gist.github.com/860354.js?file=hello-world-express.js%20">
</script>
<br />
<h3>Summary</h3>
This only scratches the surface as far as node.js goes. There are all kinds of packages available for node ones like express, view engines, low-level functionality, etc. In my project I’m using node.js as a middle man that has access to <a href="http://wiki.basho.com/">Riak</a>, via raik-js), run my code generation scripts, and manage the code that is generated be the scripts. Due to Node's simplicity, I have been able to concentrate on the functionality that I’m trying to implement and not messing with server configurations.Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com3tag:blogger.com,1999:blog-724993853864308794.post-54307093913149713352011-03-01T15:08:00.001-05:002011-03-01T20:36:57.810-05:00Instantiating a Ruby Class that is a Member of a Module from C# (a work around in 2 parts)<p>Recently I was experimenting with calling my generator ruby scripts from within an ASP.NET MVC 3 project and  I ran into a problem when I attempted to instantiate a class that is a member of a module.  I was sure others had ran into this issue so I hit the internet to see what others have done to get around this issue.  In this blog post I’ll lay out what my issue was and how I found the work around for it.</p> <h3></h3> <h3></h3> <h3>The Code</h3> <p>Here’s a scaled down version of the ruby class layout that I’m attempting to instantiate:</p> <script src="https://gist.github.com/849398.js?file=cmd_line_outline.rb"></script> <p>I took the guts out of the methods since they aren't important. The key here is that the class is in a module. The C# code that I’m trying to instantiate the ruby class in looks like this:</p> <script src="https://gist.github.com/849406.js?file=gistfile1.cs"></script> <p>I’m not going to get into the nuts and bolts of how to call into ruby from C# but I am going to give you a quick overview of what the code is doing here.  The first line creates the object we need to call into ruby. I then set up the search path so all the required files can be found.  The next step is to execute the script that has the class I want to instantiate.  Once I have the object I want to execute the run method on the object and return the results. </p> <p>Line #9 in the C# snippet is where the error was thrown.  It would complain about not being able to find the object.  I tried to use the full name, Generator::CmdLine, and just the class name CmdLine with the same result. After that I went to Google. Everything I found said that a class within a module couldn’t be instantiated from within C#. I found that a bit odd since modules are all over ruby so I turned to StackOverflow.com.  </p> <h3>The Workaround </h3> <p>I posted the question on StackOverflow,   <a href="http://stackoverflow.com/questions/5106774/instantiating-a-ruby-class-that-is-in-a-module-from-within-c" target="_blank">Instantiating a ruby class that is in a module from within C#</a>.  I figured if I’ve run across this problem someone else probably has as well. Within less than 24 hours Shay Friedman author of <a href="http://www.amazon.com/gp/product/0672330784?ie=UTF8&tag=rrowecom-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0672330784" target="_blank">IronRuby Unleashed</a> had answered my question. Here is what he suggested:</p> <script src="https://gist.github.com/849466.js?file=gistfile1.rb"></script> <p>This method is to be placed at the bottom of the cmd_line.rb file outside of the module so it is visible from within the C# code. In the C# code I change the line #9 from the C# code snippet above to this:</p> <script src="https://gist.github.com/849769.js?file=gistfile1.cs"></script> <p>Now I call the hack method from C# to get access to the Generator::CmdLine object. The work around got me past the issue I was having and I was able to continue on with my research.</p> <h3>Summary</h3> <p>Having the ability to create objects from ruby classes in C# code is nice but it is not fully functional at this point in time.  However, using resources such as <a href="http://www.stackoverflow.com" target="_blank">StackOverflow</a> can help you find ways around some of the limitations that currently exist.  I’m also thankful for people like Shay Friedman who take time out of their busy day to help the likes of me.  </p> <p>This little adventure also has me wondering how many people out there are instantiating objects from Python or Ruby in production or in their day to day work environments.  If you are leave a comment with a brief description of how you are using it and what you think about it.</p> Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com0tag:blogger.com,1999:blog-724993853864308794.post-10097313672876030552011-02-23T15:38:00.001-05:002011-02-23T17:03:49.536-05:00My Generator in the ‘Real World’–Generating Code from a JSON source<p>A while back I wrote a blog about <a href="http://rob-rowe.blogspot.com/2010/10/using-ironruby-at-work-part-2-using.html" target="_blank">using IronRuby to generate assemblies</a>.  Back then the project had a pretty narrow goal which was to read information from a group of database tables and generate C# classes to model the data describe in the three tables.  It works nicely for what I needed for my day job.  While I was working on the original project I thought it would be nice to add more input sources like text files, URLs, etc.  In addition to multiple data sources I wanted to add the ability to produce code for other languages such as Ruby.  Given the time constraints I was working in I put those additions off.  Recently I had some time free up (yeah, right!) so I created a second project that is aimed at a wider field of use, Linux, Mac as well as windows.  I have added Ruby as an output language plus a new URL source.  . </p> <p>In this blog will illustrate how a URL source that returns JSON can be used to create C# code.  The generated code will be responsible for retrieving the data and converting it from JSON to C# classes.  Next, I will be incorporated into a MVC project that I will use to display the standings from the given season.  The goal here is to show how quick you can use the code to display on the page.</p> <h4>Step 0.  Create new ASP.NET MVC 3 Project</h4> <p>Go through the usual steps of creating a new project in Visual Studio. Once you have created the project fire up the Nuget console and run:</p> <p>Install-Package Newtonsoft.Json</p> <p>The Newtonsoft.Json package is used by the service class we will generate in the next step.  The library will convert the JSON data and loads the C# class. </p> <p><em>If you don’t have Nuget you can go the </em><a href="http://nuget.org" target="_blank"><em>nuget.org site</em></a><em> or install it through the Visual Studio Extension Manager.  It is an easy way to install third-party libraries to your project.  It feels very much like installing a gem in ruby.</em>  </p> <p>Ok, we have the project the way we need it now its time to generate the classes.</p> <h4>Step 1. Generating the Classes</h4> <p>I have created a batch file to run the code generation script so I don’t have to type out all the options.  The more I type the more mistakes I make.  Anyway here is the batch file. </p> <p><script src="https://gist.github.com/840537.js?file=generator_code.bat"></script>The -i and -url parameters are used together to tell the generate that we are using URL as an input source and the –url passes the actual URL to use. The -l c_sharp and -mc switches are used to pass the desired output language and the name of the primary model class. The -mod and -sod options are used to tell the generator where to write the model and service files respectively.  </p> <p>Next step is to run the script which takes all of a second and now I have my classes.  </p> <p><a href="http://lh3.ggpht.com/_rC3qLzTABK0/TWVwOshv13I/AAAAAAAAAic/-rgSvXOBfcs/s1600-h/image%5B2%5D.png"><img style="background-image: none; border-right-width: 0px; margin: 0px 10px 0px 0px; padding-left: 0px; padding-right: 0px; display: inline; float: left; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" align="left" src="http://lh3.ggpht.com/_rC3qLzTABK0/TWVwO13GTyI/AAAAAAAAAig/dQYRCp3hNoU/image_thumb.png?imgmax=800" width="186" height="214" /></a>A total of seven classes were generated to model the JSON (<a href="https://gist.github.com/840620" target="_blank">View the JSON here</a>) that was returned.  The Standings.cs file is the primary model, meaning it is the topmost class in the source.  All other classes that end with Model were discovered in the input parsing process.  The all represent a property of the primary model or one of its supporting models.</p> <p>The <a href="https://gist.github.com/840627" target="_blank">StandingsService.cs</a> class will be used in our MVC project to retrieve the data and populate the Standings class.  We will use that class to display the data in a view.</p> <p>Now that I have my JSON based classes it is time to start creating the web front end.</p> <h4>Step 2.  Incorporating the StandingsService Class</h4> <p>Since I chose to create an empty MVC project the first thing I’m going to do is create a StandingsController class without the CRUD methods.  I also created an index view that we will use to display standings.  Here is the code that I’ve added to the controller’s index method to retrieve the standings.</p> <script src="https://gist.github.com/840931.js?file=gistfile1.cs"></script> <p>As you can see there is much to this method.  I am instantiating the StandingsService class and calling the Get method which returns an IEnumerable.  Since I know there will only be one object in the list I’m grabbing that and sending it to the view.  If this was actual production code I’d have a few more lines here to validate the year parameter, have the URL in the web.config file and check for nulls before I sent the data off to the view. Since its demo code I’m keeping it simple.</p> <p>The same goes with the view, demo simple. I sort the values by league, division and ranking.</p> <script src="https://gist.github.com/840960.js?file=gistfile1.html"></script> <p><a href="http://lh6.ggpht.com/_rC3qLzTABK0/TWVwPD6AehI/AAAAAAAAAik/FM0Eh2QW6zI/s1600-h/image%5B5%5D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; margin: 0px 20px 0px 0px; padding-left: 0px; padding-right: 0px; display: inline; float: left; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" align="left" src="http://lh5.ggpht.com/_rC3qLzTABK0/TWVwPUoWXFI/AAAAAAAAAio/H9hbrg2JWds/image_thumb%5B1%5D.png?imgmax=800" width="184" height="244" /></a></p> <p><a href="http://lh4.ggpht.com/_rC3qLzTABK0/TWVwPjPYj5I/AAAAAAAAAis/OmqVipcZ6qo/s1600-h/image%5B8%5D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; margin: 0px 10px 0px 0px; padding-left: 0px; padding-right: 0px; display: inline; float: left; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" align="left" src="http://lh6.ggpht.com/_rC3qLzTABK0/TWVwSd_LmoI/AAAAAAAAAiw/hMP0IK0weMc/image_thumb%5B2%5D.png?imgmax=800" width="161" height="244" /></a></p> <p> Nothing fancy here but going from nothing to displaying data in less than 10 minutes is a good way to start any project.  The generator takes away the tedious part of data access, creating the models and service classes and lets me get down to the business layer of the application.  </p> <p> </p> <h4> </h4> <h4>Summary</h4> <p>I started this project off with a URL to a web site that returns MLB standings in JSON format and I wanted to be able to display the results in my web site.  Using my generator project I was able to create the model classes and a service class needed to manipulate the data in C# in less than five seconds.  After the classes were generated it took me somewhere between 5 to 8 minutes to get a web page up and running that took the data from the URL and displayed it in my MVC view.</p> <p>The generator project very young and hasn’t had many real world tests.  In the near future I will be running it through more rigorous testing.  Look for more blog posts on and around the generator project.</p> <p><a href="https://github.com/rippinrobr/Examples/tree/master/BaseballMvc3" target="_blank">The MVC project can be downloaded from here</a></p> <p><a href="https://github.com/rippinrobr/Generator" target="_blank">The Generator code  (Remember, this is very young/green code)</a></p> Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com0tag:blogger.com,1999:blog-724993853864308794.post-63060777870889965252011-01-25T10:28:00.000-05:002011-01-25T10:35:50.944-05:00Using SpecFlow to test my F# Baseball Stats Library<p>As part of my Ruby indoctrination I picked up a copy of <a href="http://www.amazon.com/gp/product/1934356379?ie=UTF8&tag=rrowecom-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1934356379">The RSpec Book: Behaviour Driven Development with Rspec, Cucumber, and Friends (The Facets of Ruby Series)</a><img style="border-bottom-style: none !important; border-right-style: none !important; margin: 0px; border-top-style: none !important; border-left-style: none !important" border="0" alt="" src="http://www.assoc-amazon.com/e/ir?t=rrowecom-20&l=as2&o=1&a=1934356379" width="1" height="1" /> .  I’m about half way through the book and I find the Behavior Driven Development (BDD) process very comfortable.  Writing tests this way just ‘seems right’ and the process has already improved my Ruby code.  However, during my day job I write C# code so I started looking around to see what BDD options are available for .NET.  That’s when I found <a href="http://specflow.org/" target="_blank">SpecFlow</a>, which plays the role of Cucumber in the Ruby world.  So to get up to speed with SpecFlow I’ve decided to use it to help me test my obp function which I wrote in <a href="http://rob-rowe.blogspot.com/2010/12/project-chadwick-2top-5-sf-giants-obp-f.html" target="_blank">Project Chadwick #2–Top 5 SF Giants OBP (F# Version)</a> as I build out my baseball stats library.  Before I get started I will describe the setup.</p><h3>The Setup</h3><h4>Step 0. Create F# Library Project</h4><p>On the File Menu in Visual Studio Select New Project.  In the New Project dialog, open the Other Language option and click on Visual F# then Select F# Library.</p><p><a href="http://lh5.ggpht.com/_rC3qLzTABK0/TTn430xo4gI/AAAAAAAAAhw/nJIrA-f64MY/s1600-h/FSharpProject%5B4%5D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: block; float: none; border-top-width: 0px; border-bottom-width: 0px; margin-left: auto; border-left-width: 0px; margin-right: auto; padding-top: 0px" title="FSharpProject" border="0" alt="FSharpProject" src="http://lh4.ggpht.com/_rC3qLzTABK0/TTn442HLDDI/AAAAAAAAAh0/T_8qWrfe3vI/FSharpProject_thumb%5B2%5D.png?imgmax=800" width="598" height="187" /></a></p><p>I named this project BaseballStats.  Remove the Module.fs and Script.fsx files, I’ll add my own *.fs file when the time comes and we won’t need the Script.fsx file. That’s it for the F# project until it is time to create test test project.  </p><h4>Step 1. Create the Test Project</h4><p>Since I am using MSTest for the testing I am using a normal C# test project.  I named mine BaseballStats.AccetpanceTests.  Once the project is created I need to add the following references:</p><ol><li>FSharp.Core assembly </li>
<li>F# library project you created in Step 0, in my case the BaseballStats project. </li>
</ol><p>Delete the test class that was automatically added to the test project, I wont be using it.  <em>In real life I’d create a unit test project also lets just pretend we did that here.</em>   </p><p>Now that we have the test project created its time to install SpecFlow</p><h4>Step 3. Install SpecFlow</h4><p>If you already have SpecFlow installed then you can jump down to the NuGet section, otherwise go ahead and grab <a href="https://github.com/downloads/techtalk/SpecFlow/SpecFlowSetup_v1.5.0.msi" target="_blank">SpecFlow installer from here</a>.  I downloaded and ran the installer so that I had the file templates available in the ‘Add New Item’ dialog.  After the install I ran the command below from the NuGet Console to add the necessary DLLS to the test project.</p><pre>install-package –Id SpecFlow –Project BaseballStats.AcceptanceTests </pre><p>SpecFlow uses NUnit as its test runner by default but it can be configured to use MSTest.  Since I’m using MSTest I need to update the app.config file so it looks like this:</p><pre><specFlow>
    <!-- Possible values include NUnit (default), MsTest, xUnit -->
    <unitTestProvider name="MsTest" />
  </specFlow>
</pre><p>The setup is complete.  Its time to get on with the testing!</p><h3>The Testing</h3><h4>Step 4.  Create a Feature</h4><p>To create a SpecFlow feature file right click on the BaseballStats.AcceptanceTests project and select ‘Add…’ > ‘New Item…’ And Select SpecFlow feature file.  Name it CalculatingObp.feature.  </p><p><a href="http://lh5.ggpht.com/_rC3qLzTABK0/TT3O5lhxw_I/AAAAAAAAAh4/pFQK6v4c0HU/s1600-h/image4.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/_rC3qLzTABK0/TT3O6AvInGI/AAAAAAAAAh8/_ZtxEgWMqeE/image_thumb2.png?imgmax=800" width="571" height="137" /></a></p><p>Any file with the .feature suffix will also have a code behind file that SpecFlow will use to call the step definition methods.  The step definitions are what is run to perform the tests. I will define the steps after I have finished describing the feature .  When a new feature file is created you will see the following:</p><pre>Feature: Addition
In order to avoid silly mistakes
As a math idiot
I want to be told the sum of two numbers
@mytag
Scenario: Add two numbers
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press add
Then the result should be 120 on the screen</pre><p>Obviously this isn’t the feature I want to describe but lets take a minute to discuss it.  What the file does is describe the feature we are working in an almost plain English style.  The text under the Feature line is a narrative to help remind me what I want the the feature to do.  It has no real bearing on the code that we will use to test the feature.  </p><p>The Scenario section does have an impact on the test’s code. The Given, And, When and Then statements will be used in the step definition file and they will drive the test.  </p><p>Here’s what our feature for calculating the OBP looks like:</p><pre>Feature: Calculating OBP
In order to determine the effectiveness of a batter
As a baseball fan
I want to be able to calculate a player's On Base Percentage
Scenario: Calculate a Season's On Base Percentage (OBP)
Given A batter had "389" ABs, "104" Hits, "56" BBs, "0" HBPs,
and "4" SFs
When I run the calculation
Then I should see the result "0.356"</pre><p>The feature is used to describe how I am going to calculate a batter’s seasonal OBP.  When I save the feature file a code behind file is created that contains code that SpecFlow will use to find the step definition.  </p><h4>Step 5. Create the Steps</h4><p>Now that I have the feature description in place I’m ready to test.  Lets run the SpecFlow test and see what happens.  To run the test make sure the feature file is the selected tab in VS and click the ‘Run Test in Current Context’ button.  When you run the test the results will say ‘Inconclusive’.  View the test run details and you will see a statement that says there were no matching steps found for …. which maps to the first line of the scenario.  A little further down in the ‘Standard Console Output’ section you will see that SpecFlow has provided us with boiler plate code for the step definitions that looks like:</p><pre>Given A batter had "389" ABs, "104" Hits, "56" BBs, "0" HBPs, and "4" SFs
-> No matching step definition found for the step. Use the following code to create one:
[Binding]
public class StepDefinitions
{
[Given(@"A batter had ""389"" ABs, ""104"" Hits,
""56"" BBs, ""0"" HBPs, and ""4""
SFs")]
public void GivenABatterHad389ABs104Hits56BBs0HBPsAnd4SFs()
{
ScenarioContext.Current.Pending();
}
}
When I run the calculation I should see
-> No matching step definition found for the step. Use the following code to create one:
[Binding]
public class StepDefinitions
{
[When(@"I run the calculation I should see")]
public void WhenIRunTheCalculationIShouldSee()
{
ScenarioContext.Current.Pending();
}
}
Then I should see the result "0.356"
-> No matching step definition found for the step. Use the following code to create one:
[Binding]
public class StepDefinitions
{
[Then(@"I should see the result ""0\.356""")]
public void ThenIShouldSeeTheResult0_356()
{
ScenarioContext.Current.Pending();
}
}
</pre><p>It is time to add a step definition file to our testing project.  Right click on the AcceptanceTests project and select Add New Item Select the SpecFlow Step Definition option and give it the name ObpStepDefinitions.</p><p><a href="http://lh6.ggpht.com/_rC3qLzTABK0/TT30jQrfTGI/AAAAAAAAAiI/Oe2emmgAWcw/s1600-h/image%5B8%5D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/_rC3qLzTABK0/TT30jpT2ZhI/AAAAAAAAAiM/_5tr8t2_BiI/image_thumb%5B3%5D.png?imgmax=800" width="602" height="151" /></a></p><p>Remove the template code that is inserted into the ObpStepDefinitions test and replace it with the boiler plate methods, from the ‘Standard Output Console’ area of the test results, that were generated when when ran the SpecFlow test. My step definitions should now look like this:</p><script src="https://gist.github.com/793445.js?file=gistfile1.cs"></script><br />
<p>Run the test again and this time the test should report:</p><pre>Assert.Inconclusive failed. One or more step definitions are not implemented yet.
ObpSteps.GivenABatterHad389ABs104Hits56BBs0HBPsAnd4SFs()</pre><p>This is a good sign!  What it means is that SpecFlow now sees the steps and attempts to execute them but since the only code in the first method is the ScenarioContext.Current.Pending method call and it halts execution there since this is the first step.  Now its time I put some actual code into the method bodies.  I am going to start with the Given step. I’m going to use the numeric values in the Given statement as inputs for my test.  How can I do that?  With SpecFlow I can use regexes to grab the numeric values from the Given attribute so we can use them to run the OBP calculation.   The values grabbed by the regexes are then passed to the step method via parameters.  The values will be converted to the specified data type in the method signature by SpecFlow.  My updated step looks like this: </p><script src="https://gist.github.com/793576.js?file=gistfile1.cs"></script><br />
<p>This step is responsible for retrieving and storing the input values that will be used to calculate the OBP. Now when I run the test I see:</p><pre>Assert.Inconclusive failed. One or more step definitions are not implemented yet.
ObpSteps.WhenIRunTheCalculationIShouldSee()</pre><p>Again, it is a good sign.  It actually executed the first step and is now trying to run the When step, but it encounters the Pending method again.  This is the step where will do the OBP calculation. Since we do not have the Baseball.obp function in the F# code, we are adding code to the step that 'We wish we had', a phrase the author users repeatedly in the RSpec book. Here is the update step definition.</p><script src="https://gist.github.com/793674.js?file=gistfile1.cs"></script><br />
<p>Since the Baseball.obp function doesn't exist yet we will not be able to build the project. So I am going to switch to the F# BaseballStats project and write just enough code to allow us to build and run the test. First we create a BaseballStats.fsi file followed by a BaseballStats.fs file. In F# projects a file’s order of appearance matters in the build process.  Make sure that the fsi file appears before the fs file in the project’s listing.  To move a file up or down right click on the file you wish to move and choose the appropriate movement direction.</p><p><a href="http://lh4.ggpht.com/_rC3qLzTABK0/TT34J1LXecI/AAAAAAAAAiQ/G-YrH4E889k/s1600-h/image%5B12%5D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/_rC3qLzTABK0/TT34KP7pDYI/AAAAAAAAAiU/Z1zEDqiW70E/image_thumb%5B5%5D.png?imgmax=800" width="600" height="173" /></a></p><script src="https://gist.github.com/793781.js?file=BaseballStats.fsi"></script><br />
<p>The BaseballStats.fsi file is a signature file, you can think of it like a C/C++ header file. It describes the functions that are available in the BaseballStats.fs file. The val obp line is describing the function's signature. There will be 5 float parameters and it will return a float value.</p><script src="https://gist.github.com/793794.js?file=gistfile1.ml"></script><br />
<p>The BaseballStats.fs file is where the function is implemented. <em>In the real world we’d write just enough code to allow us to run the test again and when it failed we’d drop into unit testing or a RSpec .NET equivalent until the obp function was fully functional.  Then we’d come back to SpecFlow, run the test and get green.  In order to keep this post as brief as possible I’m not going to illustrate the process here. I have added the entire obp function but pretend we went the through process I just discussed.  </em>Once I’ve added the obp function to the fs file build the solution and run the test.   It still comes up as Inconclusive. This time it is due to the last step not being defined. </p><script src="https://gist.github.com/793710.js?file=gistfile1.cs"></script><br />
<p>The final step in my test is assert that the calculated OBP equals the expected value.  Again I’m using regex to grab the expected value which will compared against a rounded off version of the results from the Batting.obp call. Once we have green we know that the feature is working for this set of test data. </p><p>Here is what the detailed view of the test run should look like:</p><p><a href="http://lh4.ggpht.com/_rC3qLzTABK0/TT3O6iI0zsI/AAAAAAAAAiA/Yv-bzL2Dq8M/s1600-h/image%5B4%5D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/_rC3qLzTABK0/TT3O7djBpzI/AAAAAAAAAiE/TS3LZATDxJA/image_thumb%5B1%5D.png?imgmax=800" width="482" height="257" /></a></p><p>That’s it, we have green!  The OBP function performed as we had expected.  Obviously we haven’t fully tested it but we now know that the function works with valid inputs.  So what happens when we provide negative values or values that make the denominator zero? SpecFlow has a way that will allow me to use this single scenario to test all the possible permutations I can think of without writing additional scenarios.  I’m going to save that topic for a later post. </p><h3>My Thoughts on SpecFlow </h3><p>SpecFlow gives .NET developers a way to get BDD into our projects.  In the beginning using SpecFlow doesn’t seam to flow as smoothly as Cucumber and RSpec in the ruby world.  This may be due to the fact that I haven’t used SpecFlow enough or could be due to the C# and Ruby differences.  Overall, I like the BDD style of development that SpecFlow brings to the .NET world.  BDD seems to fit better to my way of thinking.  I am going to continue to use SpecFlow in my side projects and will work on incorporating it into my ‘day job’ environment.</p><h3>Resources</h3><p>You can download the source <a href="https://bitbucket.org/robrowe/specflow_f_sharp_baseballstats" target="_blank">here</a></p><p>SpecFlow: <a href="http://specflow.org/" target="_blank">project web site</a></p><p><a href="http://tekpub.com/" target="_blank">TekPub’s</a> free video on <a href="http://tekpub.com/view/dotnet-oss/3" target="_blank">SpecFlow</a></p><p>F#: <a href="http://fsharp.net" target="_blank">fsharp.net</a></p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com1tag:blogger.com,1999:blog-724993853864308794.post-17827661000904724642011-01-20T15:06:00.001-05:002011-01-20T15:16:12.747-05:00Project Chadwick Update<p>I had hoped to crank out Project Chadwick posts about once a week but as you can tell that hasn’t happened.  One of the reasons is I was a little too ambitious writing them in four different languages especially since I’m learning the languages as I go (except for ruby).  Plus I’ve been working on a few side projects that have taken more time than I had anticipated.  So, what I’ve decided to do is to continue with the series using only two languages F# and Clojure.  </p> <h4>Why these two languages?</h4> <p>Basically, I’ve become very curious about functional languages so why not use this series as a starting point.  One of the reasons I’m staying with F# is it as a CLR based language which means that I may be able use it in future projects at my day job. I’m replacing Erlang with Clojure simply because I went to a Raleigh.rb, the local ruby meetup group, meeting where they had talk on Clojure.  The speaker made Clojure sound fun so I thought I’d give it a try.  I will be using the port of Clojure that runs on the CLR simply for ease of development for me. </p> <h4>Up Next…</h4> <p>I will be adding a few more problems to the <a href="http://rob-rowe.blogspot.com/p/project-chadwick-problems.html" target="_blank">Problems Page</a> in addition to going back to the beginning and writing solutions in Clojure.  I’m looking forward to delving into the world of Clojure and functional programming in general.</p> Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com0tag:blogger.com,1999:blog-724993853864308794.post-22104465251092035672011-01-11T13:03:00.000-05:002011-01-11T13:03:03.562-05:00Project Chadwick #2–Top 5 SF Giants OBP (Ruby version)<i>Before I get started on this post if you aren't familiar with Project Chadwick here's a <a href="http://rob-rowe.blogspot.com/2010/11/introducing-project-chadwick.html">quick overview</a>. </i><em>The data for this problem can be downloaded from <a href="https://bitbucket.org/robrowe/project-chadwick/src/7aa120de3838/f_sharp/project-chadwick/data/sf_giants_batting.csv">here</a></em><br />
<br />
<h3>The Problem</h3>The On Base Percentage or OBP is the percentage of time the player reaches base. It is calculated using the following formula:<br />
<pre>OBP = (H + BB + HBP) / (AB + BB + HBP + SF)
H = Hits
BB = Walks
HBP = Hit By Pitch
AB = At Bats
SF = Sacrifice Flies</pre><p>The batter should have had at least 200 at bats to be eligible. The results should be printed out with in the following format: players name, season, and OBP.</p><br />
<h3>The Solution</h3><script src="https://gist.github.com/774717.js?file=gistfile1.rb"></script><br />
<br />
<h3 style="margin-bottom: .35em;">New Ruby Concepts</h3><h4>Ruby Class</h4><span class="Apple-style-span" style="font-size: small; font-weight: normal;">The first ‘new’ thing you see in this script is a class. There are three items I'd like to discuss about this class: attr_accessor, initialize and to_s methods.</span><br />
<br />
<script style="margin-bottom: .35em;" src="https://gist.github.com/774477.js?file=gistfile1.rb"></script><br />
<span>The attr_accessor is a method that is used to indicate which instance variables, also known as attributes, are available outside of the class. All of the variables declared in the attr_accessor call can be read and updated from outside of the class. You can also set up attributes that are read-only by calling attr_reader. To set up a write-only attributes by calling the attr_writer. When attributes are used within a class they are prefaced with @ sign.</span><br />
<br />
<span>Another feature of a Ruby class is the initialize method which is the class’s constructor. In this class there isn’t much going on here other than taking the stats and putting them in a variable that makes it easier to read the code.</span><br />
<br />
<script style="margin-bottom: .25em;" src="https://gist.github.com/774588.js?file=gistfile1.rb"></script><br />
<br />
<span>Well, there is a little more going on then a simple assignment. In the stats fields the ||= says that if the value on the left is nil then assign 0.0 to it. The value is then being converted to float so we can do division.</span><br />
<br />
<span>The last method I want to discuss is the to_s method. This method is overriding the base class’s to_s which is the a to string class. In this method I’m just formatting the output of the class so my loop is a little cleaner.</span><br />
<br />
<script src="https://gist.github.com/774795.js?file=gistfile1.rb"></script><br />
<h4>Arrays</h4><span>One of my favorite things with Ruby is how arrays can be manipulated and used. The first example I’m adding the newly created Batter object to the end of the all_obps array. In the second line I'm sorting the array by the obp value. What I'd like to point out here is the ! at the end of sort. The ! indicates that the sorting will be done in place, in other words it changes the value of the array I’m sorting. If I left the ! off the sort would create a new array that was sorted leaving me with the original array AND a new, sorted array. Finally, the reverse! line has the [1,5] added to it. Which means I want the first through the fifth elements. In our case that is how I grab the top five OBP for the SF Giants.</span><br />
<br />
<h3>An Overview of My Solution</h3><br />
<span>In this script I’ve opted to write code that was easier to read instead of the fewest lines that’s why I created the Batter class. This way we can access the stats needed to calculate the OBP in a more straight forward way. Since I’ve already pointed out some of the nuances of Ruby classes and the calc_obp method being straight forward I’m not going to going to dissect the class any further here.</span><br />
<br />
<script style="margin-bottom: .35em;" src="https://gist.github.com/774686.js?file=gistfile1.rb"></script><br />
<br />
<span>What I will talk about how the script reads and processes the data. The first line in the snippet reads all lines of the file and then loops over it. I then create the Batter object and check to see if the batter had at least 200 at bats. If not I move on to the next line. If the batter had at least 200 at bats I calculate the OBP and store the Batter object in the all_obps array. The rest of the script sorts, reverses and prints the top five OBP for the San Francisco Giants.</span><br />
<br />
<span>That’s it, nothing to the Ruby script when you compare it to the <a href="https://gist.github.com/741374#file_gistfile1.fs">F# script</a>. This may be due to the fact that I’ve worked with Ruby more than I have F# but I believe Ruby is just a little more concise than the F#. I may change my mind after writing more F# code but for now that’s the way it seems to me.</span><br />
<br />
<span>I really enjoy writing Ruby code, it allows me to concentrate on what I need to do more than 'how do I do this in Ruby'. I guess what I'm trying to say it feels more natural than other languages such as C#.</span><br />
<br />
<h4>Up Next...</h4><br />
<span>I'm still working on getting more problems up for those of you who are interested in continuing on the baseball stats path. I'll write the Erlang version next and then wrap up the second problem with the Objective-C code.</span> <br />
<p>Thanks for stopping by.</p>Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com0tag:blogger.com,1999:blog-724993853864308794.post-42444624922831692422010-12-12T14:27:00.001-05:002010-12-15T11:33:21.777-05:00Project Chadwick #2–Top 5 SF Giants OBP (F# Version)<p><i>Before I get started on this post if you aren't familiar with Project Chadwick here's a <a href="http://rob-rowe.blogspot.com/2010/11/introducing-project-chadwick.html">quick overview</a>.</i></p> <p><em>The data for this problem can be downloaded from <a href="https://bitbucket.org/robrowe/project-chadwick/src/7aa120de3838/f_sharp/project-chadwick/data/sf_giants_batting.csv" target="_blank">here</a></em></p> <p>In this problem we are going to find the Top 5 On Base Percentage Seasons since the Giants moved to San Francisco.  While I’m not convinced that I’m fully thinking like a functional programmer but I think I’m starting to ‘get it’.  With that said, lets get started.</p> <h3>New F# Concepts</h3> <p>In my second F# script, I’m using a few concepts that I didn’t use in the first solution, namely <strong>record</strong> and multi-lined<strong> functions</strong>. </p> <h4>The Record Type</h4> <p><code>type Batter = { Last : string; First : string; Season : int; AB : float; OBP : float }</code></p> <p>So what is a record?  It looks like a different way to declare a class.  Well, it may look that way but records are not the same as classes.  Records allow you to group data into types and access the data using fields. Fields in records are immutable whereas classes do not offer the same type safety.  Also, records cannot be inherited.  There are other differences that are beyond the scope of my current F# knowledge but as my F# knowledge expands we may delve into the remaining differences.</p> <h4>Functions</h4> <p><script src="https://gist.github.com/738218.js?file=gistfile1.fs"></script>A function is declared using the let statement. The keyword let is followed by the name of the function, a list of space delimited parameters, and optionally a return type.  The body of the function is determined by white space.  All lines that are indented after the declaration are considered part of the function’s body until a line is encountered at the same ‘level’ of indention as the let statement. The return value of a function is the result of the last line executed.</p> <h2></h2> <h4>Seq.toList, Pipe Forward Operator and List.Map</h4> <p>When we read in the content of the data file using the ReadAllLines method the file content is returned as a string array.  In F# it is easier to work with lists in than arrays, least with what F# knowledge I have.  So to convert our file content into a list I used the Seq.toList method. The pipe forward operator, |>, is used to send the output of the ReadAllLines call to send or ‘pipe’ it to the Seq.toList method as its parameter. You can think of the pipe forward operator similar to the pipe utility UNIX command line.  The results of those commands are stored as a list<Batter> in the stats variable.</p> <p><code>let stats = File.ReadAllLines(@".\data\sf_giants_batting.csv") |> Seq.toList </code></p> <p>The List.map operation allows us to take a list and pass each item as a parameter to a function in one line.  In addition to calling the function it creates a new list with the results of each column, in this case we aren’t using the returned list.  In my solution I’m using it like a one line foreach call.  I’m calling the create_batters function passing in the tail of the stats list that I read in. Why just the tail, because the head is the line that contains the column headings. </p> <p><code>List.map create_batters stats.Tail</code></p> <h3>An Overview of My Solution</h3> <p>Since I’m just getting started with F#, I find myself writing F# code that looks like C#.  I think my functions reflect that.  However, the last line makes me think that I’m starting to make the turn on understanding functional programming and how I can use it.  Here is the last line: </p> <p><code>List.map print_batter (Seq.take 5 all_obps |> Seq.toList)</code></p> <p>In my first go round of this script I had a for loop that went from 0 to 4 to print the first five items in the list.  As I was writing this post up I started looking at the loop thinking I could improve it.  I remembered reading about the Seq.take method that takes the number of  items specified from the given sequence.  So I removed the for loop and plopped in the following code in its place:</p> <p><code>List.map print_batter (Seq.take 5 all_obps)</code></p> <p>When I ran the new and improved script I received the following error:</p> <p><code>chadwick-2-top5-obp.fsx(64,24): error FS0001: This expression was expected to have type Batter list but here has type seq<'a></code></p> <p>I noticed that the error message specifically stated that it was given a sequence but it expected a list.  So I tacked on the |> Seq.toList call and was able to get it to work.  That type of code is what gets me excided about functional programming.  I’m looking forward to getting to the point where I can truly use the functional programming aspects of F#. </p> <p>Here is my entire solution:</p> <script src="https://gist.github.com/741374.js?file=gistfile1.fs"></script> <p>I enjoyed working on this solution, while its nothing big in the grand scheme of things but it was my first ‘real’ F# script.  As always, any critiques, nudges or hints would be greatly appreciated.  My number one goal of going through this process is to learn the 4 languages.</p> <h3>Up Next…</h3> <p>I will be adding more problems to the list shortly and solving this problem in either ruby or objective-c next.  </p> Robhttp://www.blogger.com/profile/04054382840301560558noreply@blogger.com0