Project Chadwick #2–Top 5 SF Giants OBP (F# Version)

Before I get started on this post if you aren't familiar with Project Chadwick here's a quick overview.

The data for this problem can be downloaded from here

In this problem we are going to find the Top 5 On Base Percentage Seasons since the Giants moved to San Francisco.  While I’m not convinced that I’m fully thinking like a functional programmer but I think I’m starting to ‘get it’.  With that said, lets get started.

New F# Concepts

In my second F# script, I’m using a few concepts that I didn’t use in the first solution, namely record and multi-lined functions.

The Record Type

type Batter = { Last : string; First : string; Season : int; AB : float; OBP : float }

So what is a record?  It looks like a different way to declare a class.  Well, it may look that way but records are not the same as classes.  Records allow you to group data into types and access the data using fields. Fields in records are immutable whereas classes do not offer the same type safety.  Also, records cannot be inherited.  There are other differences that are beyond the scope of my current F# knowledge but as my F# knowledge expands we may delve into the remaining differences.

Functions

A function is declared using the let statement. The keyword let is followed by the name of the function, a list of space delimited parameters, and optionally a return type.  The body of the function is determined by white space.  All lines that are indented after the declaration are considered part of the function’s body until a line is encountered at the same ‘level’ of indention as the let statement. The return value of a function is the result of the last line executed.

Seq.toList, Pipe Forward Operator and List.Map

When we read in the content of the data file using the ReadAllLines method the file content is returned as a string array.  In F# it is easier to work with lists in than arrays, least with what F# knowledge I have.  So to convert our file content into a list I used the Seq.toList method. The pipe forward operator, |>, is used to send the output of the ReadAllLines call to send or ‘pipe’ it to the Seq.toList method as its parameter. You can think of the pipe forward operator similar to the pipe utility UNIX command line.  The results of those commands are stored as a list<Batter> in the stats variable.

let stats = File.ReadAllLines(@".\data\sf_giants_batting.csv") |> Seq.toList

The List.map operation allows us to take a list and pass each item as a parameter to a function in one line.  In addition to calling the function it creates a new list with the results of each column, in this case we aren’t using the returned list.  In my solution I’m using it like a one line foreach call.  I’m calling the create_batters function passing in the tail of the stats list that I read in. Why just the tail, because the head is the line that contains the column headings.

List.map create_batters stats.Tail

An Overview of My Solution

Since I’m just getting started with F#, I find myself writing F# code that looks like C#.  I think my functions reflect that.  However, the last line makes me think that I’m starting to make the turn on understanding functional programming and how I can use it.  Here is the last line:

List.map print_batter (Seq.take 5 all_obps |> Seq.toList)

In my first go round of this script I had a for loop that went from 0 to 4 to print the first five items in the list.  As I was writing this post up I started looking at the loop thinking I could improve it.  I remembered reading about the Seq.take method that takes the number of  items specified from the given sequence.  So I removed the for loop and plopped in the following code in its place:

List.map print_batter (Seq.take 5 all_obps)

When I ran the new and improved script I received the following error:

chadwick-2-top5-obp.fsx(64,24): error FS0001: This expression was expected to have type Batter list but here has type seq<'a>

I noticed that the error message specifically stated that it was given a sequence but it expected a list.  So I tacked on the |> Seq.toList call and was able to get it to work.  That type of code is what gets me excided about functional programming.  I’m looking forward to getting to the point where I can truly use the functional programming aspects of F#.

Here is my entire solution:

I enjoyed working on this solution, while its nothing big in the grand scheme of things but it was my first ‘real’ F# script.  As always, any critiques, nudges or hints would be greatly appreciated.  My number one goal of going through this process is to learn the 4 languages.

Up Next…

I will be adding more problems to the list shortly and solving this problem in either ruby or objective-c next. 

Project Chadwick #1- Willie McCovey’s Career Batting Average

Before I get started on this post if you aren't familiar with Project Chadwick here's a quick overview.

I have started this off with an easy problem, calculating Willie McCovey’s career batting average.  Starting this way made it easy for me to concentrate on learning enough to get something running without taking up too much time.   In this post I will tackle a solution in all four of the languages I’m interested in:  Erlang, F#, Ruby and Objective-C.  As time goes on and the code for each solution grows I will probably move to a single post per language for each problem. 

A little background on my experience with the languages I have chosen.  I have been using Ruby for a few months now.  We have converted our build process from MSBuild over to rake and albacore, used rails to do a proof of concept, and written other Ruby scripts to do administrative things.  However with Erlang, F# and Objective-C I am truly learning them as I go through this process.  If you see room for improvement in any of my solutions please feel free to share them.

As for my setup for creating these solutions I am running the F# on Windows 7 and all other languages on Linux.  The F# may also move to Linux once I get my F# environment setup there.  With that said lets get started!

Batting Average Formula: Hits/At Bats

Languages: F#, Erlang, Ruby and Objective-C

Data: download from here.

Without further ado here are the functional languages (F# and Erlang) solutions:

Erlang Solution

A quick overview of the syntax you see in the script. First %% are used to write comments.  Erlang requires all variables start with a capital letter.  Also once a variable has been assigned a value it cannot be changed. Function bodies are preceded by the –> sign. For functions that have multi line bodies commas are used to indicate an end of line.  The main function is an example of this.  The last line of the function’s body has a period as its last character.

The first line of the file is setting up so I can run this like any other script in Linux.  After the comment lines the sum functions are defined.  Each function is uniquely identified by the module it appears in, the name and its ‘arity’.  What is arity? Arity is the number of arguments the function has.  In our cause the first sum method has an arity of 2 and the second has an arity of 1. The sum functions are two distinct functions they have nothing to do with each other.  The sum functions are used to total up the career hits and at bats.

The main function  does what you’d expect it to do, it is where the batting average is calculated.  The first two code lines set up lists of the career hits and career at bats for McCovey.  I had wanted to use tuples here but I was not getting the correct average using it.  So I junked the tuples and went with something I knew would work.  The last line of main simply prints out “Willie McCovey’s career bating average is 0.270” .  The place holder is replaced with the result of sum(Hits)/sum(Abs).  The function signature for the io:format method looks like this:

io:format([IoDevice,] Format, Data) 

The io is what module houses the format function.  format’s parameters are: IoDevice if left out stdout is used. If we were writing to a file we would pass the file handle as the IoDevice. Format is the string with as many placeholders as you need.  The Data parameter is used to replace the placeholders.  The number of items in the Data list must equal the number of placeholders in the string.

The Erlang solution was quick and to the point 

The F# Solution

F# is a .NET functional language.  I am learning this language in hopes that I can use it to do some more complicated comparisons quicker than I could in C#.  With that said, lets jump into the F# solution.

As you can see it is very similar to the Erlang code.  With F# we define the sum function in one line but it handles the same two situations as the Erlang sum functions do.  Head and tail have the same meaning as H and T do in Erlang. It adds the head to the result of the recursive call on sum.  When sum is at the end of the list or receives an empty list it returns zero. A real difference between the Erlang and F# code is that we have to add the rec keyword to the function’s definition to indicate that this is a recursive function.  The printfn method is called to write the results out to stdout.  Notice that in the calls to sum we do not use parenthesis that is because parenthesis are used for precedence operations, to create pairs and tuples and to denote a parameter of type void.  It takes a little getting use to not using parenthesis in function calls but after you do, the code seems a little cleaner.

The Ruby Solution

Now that we have the functional languages done lets move to something a little more familiar to most of us, object oriented programming.  Although this Ruby solution really doesn’t do much with objects. 

For this script I’ve combined the hits and at bats for each season as an array.  So the hits_ab array is an array of arrays.  Next I initialize a variable to store the total hits and one for the total at bats.  These are set to 0.0 so that when I do the division I do not have to convert the sums to floats using the to_f method. 

The each loop is one of the cool features of Ruby, the use of blocks.  The do …end is a block of code that is passed to the each method as a parameter.  In our case the block is taking each of the arrays that are ‘in’ the hit_ab array and storing the first entry in the h variable and the second entry in the ab variable. 

The last line prints the results to stdout after formatting the average.  The #{} in a ruby string is how you put the value of a variable, method call, or in our case the formatted results of the division into a string. 

The Objective-C Solution

Ok, I have to admit this up front, I’m really learning this language as I go through this process.  This is my weakest of the 4 languages here, so please help feel free to guide me into the proper way of doing things if you see something wrong.

Most of this solution is really straight C code but it gets the job done.  The first Objective-C or obj-c related line is the #include<Foundation/Foundation.h>.  This header file is the base header file for obj-c. The second obj-c line is the NSAutoreleasePool line.  The NSAutoreleasePool object is to support Cocoa’s, an Apple development framework, reference counted memory management system.  All of the objects in the pool are sent a release message when the NSAutoreleasePool' object is sent the drain message.  In the above code the drain message is sent in the [pool drain]; line.

More on the NSAutoreleasePool line

In this line of code NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; there are some interesting things going on.  Lets look at the inner bits of code [NSAutoreleasePool alloc].  In obj-c method calls are done like so [object methodname].  What inner call is doing is allocating the memory that the *pool pointer will point to.  When the new object is returned we call its init method which is its constructor call.  Now we have a fully initialized NSAutoreleasePool object.

 

Summary

Admittedly this was an easy problem to solve so the code for each language was very similar. It also hints that the functional languages tend to require less code to do the same thing.  Ruby is in between the functional and Objective-C code.  I’m sure I could make the Ruby solution look even more like the functional languages if I put more time into it.

Remember, I am learning these languages as I go so if you see something I’m doing completely wrong or not as gracefully as I could don’t hesitate to pass it along.  I would greatly appreciate it.  As these problems get more complicated and the code required grows, I will do each language one at a time.

Up next: I will find the top 5 on base percentages since the Giants moved to San Francisco.

Language Resources

For additional information on these languages you can visit these sites:

Erlang:        http://www.erlang.org/
F#:              The F# Survival Guide
Objective-C: Learning Objective-C: A Primer
Ruby:          http://www.ruby-lang.org/

Introducing Project Chadwick

Project Chadwick is a set of sabermetric/baseball statistics formulas I am using to learn a few new programming languages: Erlang, F#, Objective-C and Ruby.  This project was inspired by Project Euler which is a series of math problems designed to keep your math and computer skills sharp.

Why Baseball Statistics?

Because I’m a baseball fan.  As a kid I spent many hours reading box scores and whatever other stats the papers would publish. I have been curious about these languages so I thought I would use something I enjoy to learn a these programming languages.

How it will work

I will present the problem and formula I am solving/using and then give a walk through of the code.  For the smaller/easier problems I will show multiple languages at once.  When the code gets long or the formulas are more involved I’ll have a post per language. 

Remember, I’m doing this to learn these languages so if you have any tips and/or hints please feel free to pass them along

Why is it called Project Chadwick?

It is named after the person who created the baseball box score. Henry Chadwick was born in England and was a Cricket report who started reporting on baseball in 1857.  For more information on Henry Chadwick check out his wiki page or for more history on baseball statistics check out The Numbers Game by Alan Schwartz.

My Sources

One of the main sources I have for my formulas is the book Baseball Hacks by Joseph Adler.  The statistical data I will use comes from the http://www.baseball-databank.org/.  It contains statistics from 1871 to 2009. You can download a MySQL database or files to load into Excel from the databank site.  If you do not wish to store the data yourself you can view the data at  http://www.baseball-reference.com/.   

The Languages

If you want to use the same languages I am you can download and install them from the following places:

• Ruby: http://www.ruby-lang.org/en/downloads/
• Objective-C:  I’m not sure if its possible to run it on windows.  I will be writing my objective-c code on Ubuntu.  Here are the instructions for setting up Ubuntu for Objective-C.
• Erlang: http://www.erlang.org/download.html
• F# http://msdn.microsoft.com/en-us/fsharp/cc835251.aspx
  

I have the first problem posted here http://rob-rowe.blogspot.com/p/project-chadwick-problems.html.  Over the next few days I will be adding the problems and my solutions.  This should be a fun process I hope you join me in this project. 

Part 3 – Creating the Domain class with CodeDom and IronRuby

• Part 1 – Using EF with IronRuby to retrieve data
• Part 2 – Using AssemblyBuilder and System.Reflection.Emit to create a .NET Assembly
• Part 3 – Creating the Domain class with CodeDom and IronRuby.

This is the final post in a three part series in which I discuss how I used IronRuby to generate data access code.  In this post I’m going to discuss how IronRuby was used to generate a C# domain class for each model.

All source files can be downloaded from here

Generating the Domain/Service Layer Code

Now that we have models to truck the data from the DAL to the presentation layer we need the code to move the data between the two layers.  In this version of the project I used the CodeDom to generate the C# code.  This will change as I move this project more into our production environment.  I plan on moving towards a templating engine such as Ruby’s ERB or ASP.NET MVC 3 Razor view engine.  In the beginning of the project I had intended to write this layer using the Emit approach but it found it to be overkill and at this layer the chances are better that we will need to modify the generated code which is not possible if we go down the Emit path.  With that said lets dive into the CodeDom approach. 

In this post I will walk you through how I created the class, added a private field, the constructor

System.CodeDom – The Setup

In order generate the code we must first create a CodeCompileUnit object.  Think of it as a container for the code tree we are about to create.  Next we CodeNamespace object passing in the namespace that will contain the class we are creating. The last bit of setup we’ll do is to import any namespaces our class will need using the CodeNamespaceImport object.Here’s what the setup code looks like:

Creating the Domain Class

Now we can actually create our class type using the CodeTypeDeclaration class passing in the name of the type we are creating.  This call creates a type but we still need to indicate we are creating a class by setting the CodeTypeDeclaration.IsClass property to true.  We’ll also make the class public by setting the TypeAttributes property to TypeAttributes.Public.  After creating the class type we’ll add it to the namespace by passing the type to the @nspace_holder.Imports.Add method.  The code for creating the class type is below.

Adding a Field

Our class will need a field to store the reference to the repository it will use. The first step is to create a CodeMemberField object and set the attributes to private, give it the name _repo and set the type to be IRepository.  Next we’ll add it to our class by adding it to the Members list.

Adding the Constructor

Once we have the field to hold the repository we need to set it to something.  The constructor will have one parameter IRepository repo.  The body of the constructor will have a check to ensure that the repo parameter is not null.  If it is null it will throw an ArgumentNullException.

Creating the constructor object is as simple as create a new CodeConstructor object.  We make the constructor public by setting its attributes to MemberAttributes.Public.  Next, the IRepository parameter is added to the Parameters list by creating a CodeParameterDeclarationExpression object passing in the type and name of the parameter. 

Once we have created the parameter we need to grab a reference to it so we can use it to set the class’s _repo field to the value of the parameter. We pass the parameter and field references to the CodeAssignmentStatement constructor.  This object will be used as the true statement in the if/else code which is created when we call the create_if_else_statement method.

As you can see this method is pretty straight forward.  You pass in a condition to test, the statements to execute if the condition is true or false.  It returns the statement that we will add to the constructor object’s Statements list. The last step is to add the constructor to the class's Members list.

Creating the Get Methods

The domain class has two Get methods. The first one returns an IQueryable that when executed would return all records. The second will return the record that matches the Id parameter that is passed in.

All methods we create are started by calling the basic_method.  This method instantiates a CodeMemberMethod object, setting the Attributes to be public method, the name of the method and the return type.

The first get method

This method will return an IQueryable.  In order to set that up we create a CodeTypeReference object passing in IQueryable<T> where T is the EF model type to the constructor. Our next step is to add the return statement to the method’s body.  This is done by creating a CodeMethodReturnStatement passing the results of the create_repo_method_call method to it’s constructor.

The create_repo_method_call is a way to create calls to the repository class.  It creates a CodeMethodInvokeExpress object that represents the method we will call in our method.  It takes 3 parameters.  The first is a CodeTypeReferenceExpression which in our case is the _repo field. The second parameter is the name of the method we are going to call, in this case its Get. The final parameter takes an array of parameters that the called method receives.  If there are no parameters it is an empty array.

After the create_repo_method_call returns and the results of the call are stored in the method’s Statements property the method is added to the class’s Members list.

The Second get method

The second Get method takes an Id parameter which is used to find the single record that matches it. After the method’s CodeMemberMethod object is created the first thing we do is add the Id parameter following the same process we did with the constructor and create a reference to it.  Next, a MethodInvokeExpression object is created to make the call to the repository’s Get method.  In addition to this call there will be two other method calls chained to it.  The first is a LINQ Where method that takes a CodeSnippetExpression object as its parameter.  Finally an FirstOrDefault method is added to the chain.  Here is the code for the add_get_methods.

The Complete Domain/Service Generated Through this Process

Summary

Now we have the domain/service layer classes to go with the models we created in Part 2. The C# classes have methods to Get, Add, Update, Delete records that map to the models we’ve created.  These classes also have methods to map between the EF models and our models.  Generating these two layers of code and adding them to our MVC projects gives us the potential to have basic application up and running quickly.

What’s next with this project?  On the System.Emit portion of the project I will be adding the ability to store data from one model class into multiple tables, add attributes to the properties, and a way to generate models from non-database data sources such as flat files, URLs and HL7 messages.  I will continue to generate the models using the System.Emit process.  I will be changing my approach on how the source code is generated to use either Ruby on Rails’ ERB view engine or the new Razor view engine in ASP.NET MVC.  I believe this approach will make it easier for us to make changes to the process and makes it easier to maintain in the long run. I will be adding the ability to generate unit tests, controller class boiler plates and perhaps HTML views as well.

I enjoyed working on this series and I hope you enjoyed it! 

My CodeCamp RDU 2010 Experience

This past Saturday was my first CodeCamp and my first time speaking at an event.  My talk covered a project I’ve been working that uses IronRuby to generate a data access layer and domain layer classes.  The slides can be found here.  The experience of giving a talk is something everyone should do at least once.  It helped me get a better understanding of the subject I was speaking about and a new perspective on what people go through to put a talk together.

Since my talk was in the afternoon I spent the morning going to talks.  The first talk I attended was titled ‘Unit Testing for the rest of us’.  It was a nice introduction to unit testing and why you should incorporate it into your projects.  You can check out the slides here.  Eli did a great job with the talk.

The next talk was ‘Dynamic Programming in a Statically Typed World’ by Jim Wooley.  The talk wasn’t quite what I thought it was going to be but I still learned a few things. The thing about attending any talk is you will always learn at least one thing that you can apply.  The next day I was working on a personal project and I remembered the dynamic keyword demo that Jim gave and used it to resolve a sticky problem. 

Since lunch was right after Jim’s talk a co-worker and I ate shared a table with Jim.  We had a great conversation around writing your own IQueryable implementation and a little bit about natural language processing.  A definite positive to attending local talks is the availability of the speakers. After most talks they are open to continue any discussions that may have occurred during their talk.

After lunch I went to Mike O’Brien’s talk ‘What is F# and Why Should I Care?’.  It was a nice introduction into functional programming using F#.  The last talk I attended was also given by Mike.  It was titled ‘Release Management with Go/Ruby/Rake/Albacore’.  A little short and not as much on the ruby portion of the talk which is what I was hoping for.  However it was a nice introduction into Go.  The talk also spawned a conversation about other CI servers out there, such as TeamCity..

Overall it was a good experience.  I met a few people and had nice discussions.  Like I said, I have already used information I gained on Saturday.  If there is a CodeCamp in your area I would recommend attending and/or speaking at it.

The slides from my talk are here.  The source is here.

Using IronRuby at Work: Part 2 – Using AssemblyBuilder and System.Reflection.Emit to create a .NET Assembly.

• Part 1 – Using EF with IronRuby to retrieve data
• Part 2 – Using AssemblyBuilder and System.Reflection.Emit to create a .NET Assembly
• Part 3 – Creating the Domain class with CodeDom and IronRuby.

I will be giving a talk on this subject at this subject on November 6, 2010 at the Triangle .NET User Group’s RDU Code Camp

This is the second post in a three part series in which I will discuss how I used IronRuby to make my life easier.  In this post I’m going to discuss how IronRuby was used to generate a .NET Assembly.

All source files can be downloaded from here

Generating Non-EF Models

Now that I can make queries against the database I want to make non-EF models that I can use to move data from the data access layer to the presentation layer.  Some may think this is over-kill but I like to keep EF out of my presentation layer.  Plus, if I can generate these models it doesn’t take much to have this extra layer.

The code generation process is managed by a class called CodeGen.  The CodeGen class has methods to generate the models and the domain classes.  CodeGen’s initialize method (ruby class’ constructor) there are three important objects created: GenerateAssembly, ModelGenerator and ElementsService.  The GenerateAssembly class is where the actual assembly is created.  The  initialize method creates the dynamic assembly and a module are created with these two calls:

The first call creates the dynamic assembly by passing in the assembly name and version information which is stored in the @asm variable.  The AssemblyBuilderAccess.RunAndSave is used to indicate that we want the ability to run the code as well as save it to the file system. The second call creates the module for the assembly.  I will use the @mod_builder object to create our model classes for our assembly.

After I have a CodeGen object I will call the create_models method.

The create_models method uses the ElementsService class (discussed in the first post of this series), to retrieve all active views using LINQ in my IronRuby script.  For each view returned a call to create_model is made.

The create_model method is where we actually start generating CIL code.  The @model_gen.generate method drives the actual model class creation and adds it to the assembly.  It returns the actual CLR type of the new model and an array that contains the names of all domain level classes that will be used to drive the domain class generation process.  Let’s get into the actual generation of the model classes.

The ModelGenerator class

The ModelGenerator.generate method is called with two parameters.  The first parameter is the name of the class to create and the second one is an array of properties to create.  

The first line of code starts the generation by calling GenerateAssembly.define_class method which returns a TypeBuilder object that represents the class.  The GenerateAssembly.define_class method looks like this:

The define_class parameters are pretty straight forward.  The first parameter is the name of the class to be created.  The second parameter contains the name of the interface the class is going to implement.  If no name is given it will be set to nil (null in C#). The @mod_builder variable is a ModuleBuilder object which I will use to create the model class.  Calling the DefineType method creates a representation of the model class which is returned in a TypeBuilder object that we will be used to create the class’ properties. 

When the define_class call returns we will create the properties.  Creating properties is a little more involved then creating a class.  The properties will be created using the ModelGenerator.create_property method.  The create_property method takes the TypeBuilder object that represents the class, the name of the property and the CLR type of the property.  The last parameter is optional and I will go with the default.

The first line creates the private field that will store the value of the property using the TypeBuilder to call the DefineField method.  I keep the returned FieldBuilder object so I can use it in the body of the setter/getter methods. 

Next I will create the property by calling TypeBuilder.DefineProperty.  The parameters are the name of the property, attributes that describe the property, the CLR type and the parameter types.  Since I do not have parameters in our properties I set it to nil.  The returned PropertyBuilder is stored so I can use it to associate the getter/setter methods to it.

The next step is to create the getter method by calling TypeBuilder.DefineMethod object.  After the DefineMethod call I can start generating CIL, also known as MSIL.  The DefineMethod call returns a MethodBuilder object that we use to get an ILGenerator object which is how I will create the body of the getter method.  The generation is done by making calls to the Emit method.

The first Emit call loads the argument at index one onto the stack.  The next line finds the value of a field in the object whose reference is currently on the evaluation stack, in this case it is the private field that I created first.  The last line adds the return statement to the getter.  As you can see there isn’t much to setting up the getter method.

The process for the setter is similar to the getter except for two things.  The first difference is setting up the parameters for the setter.  The second is to set the value of the private field.  The parameters for the setter are created by these lines. 

Since I am using IronRuby I had to create a generic list of the type Type and add the type parameter to the list. There may be another way to do it but I don’t know any other way to get a CLR array of Type.  Just prior to calling the DefineMethod I convert the list to an array using the .NET generic list’s ToArray method and pass it to the the DefineMethod as the last parameter.  This sets the setter method’s parameter type.

The second difference is the section of code that store the parameter’s value in the private field.  The Ldarg_1 line loads the parameter onto the stack.  The next line sets the value our private field to the value of the parameter.  All the rest of the code for the setter is the same as the getter method.

Finally I relate the getter and setter methods to the property by making these two calls below.

Back to ModelGenerater.generate

After creating all the properties for the model the next thing we need to do is to create the Type for the model’s class. Which is done by making the GetType call below.

Once the GetType method is called I am done with the creation of the model’s class.  After all of the models are created the script writes the new assembly out to the file system using the AssemblyBuilder object that was created in the first line of the CodeGen class’ initialize method.

Summary

After reading the information from the database I was able to convert that data into a model classes in CIL without much work.   I can now take information from 3 database tables and generate models for the classes we need in a few seconds.  Now that I have the generated models and EF entities I am almost ready to start using the data.  In Part 3 of this series I will go over how I create the domain layer classes using the CodeDom.

You can download the IronRuby and C# code for this series from here.

Microsoft turns IronRuby and IronPython over to new Leadership

You can read the official announcement here.  In addition to the turnover they now have visual studio tools for IronRuby.  The tools can be found here.  Install the IronRuby 1.1.1 it has the tools for visual studio. I haven’t used them yet but thought I’d let you know.

Getting Rails 3 Up on Windows Connecting to SQL Server

Our web group is starting a few test projects on Ruby on Rails for some of our tools.  Our group works in the windows environment with both client/server and ASP.NET MVC apps.  This morning I went through the process of getting two machines set up to do Rails development and I thought I’d write up a quick post to pass along what I learned.  First off I want to say it wasn’t as painful as I thought.  Here is what I did to get rails 3 up and running against SQLServer 2005 and 2008.  My environment is windows 7 64 bit.  I also installed it on Windows 7 32 bit.  For the purposes of this post I will walk you through my install on the 64 bit OS.

I am assuming you have a clean (no ruby installed) machine and a new/clean database to work with.  If you already have ruby and some gems installed you can skip some of the steps.  However, you must have the RubyInstaller version of the interpreter installed for this to work.  With that, here are the steps to running rails 3 on Windows and SQL Server!

Step 0 – Setting up a DSN

For 32 bit Windows:  Go to Control Panel > System And Security > Administrative Tools > Data Sources (ODBC) and create a System DSN (I’m not sure if It has to be a System DSN but that’s what I did).  For the purposes of this blog post I choose the With Windows Integrated Authentication option for authentication.

For 64 bit Windows:  First and foremost, if you setup your DSN following the 32 bit approach you will not be able to connect to the database with rails.  You will receive an error like this: 

ERROR [IM014] [Microsoft][ODBC Driver Manager] The specified DSN contains an
architecture mismatch between the Driver and Application

The above error is due to the fact that you have set up a DSN with the 64 bit version of the Data Sources applet.  After a little quick Google search I found this fix on Stack Overflow.  Basically, you have to run the version at c:\windows\sysWOW64\odbcad32.exe to setup the DSN. After the DSN configuration lets move on to install ruby and all the necessary gems.

Step 1 – Installing Ruby

The first thing you need to do is install ruby using the RubyInstaller which can be found at RubyInstaller.org (download link for 1.9.2).  This method of installing ruby is required or you will not be able to do step 2. 

Step 2 – Install the RubyInstaller DevKit

After you have installed ruby the next step is to download the devkit from from the rubyinstaller.org here.  Follow these instructions explicitly and you will be ready to start installing the database related gems.

Step 3 – Installing the Database Related Gems

Open a command prompt if you do not already have one open and run the following commands:

1. gem install ruby-odbc 
2. gem install activerecord-sqlserver-adapter

Step 4 – Install Rails and Create a Test Application

Once you have the DB gems installed the next step is to install rails itself:

• gem install rails

When the install is complete change and/or create a directory where you want to create the test application.  Once you are in the directory where you want the application run the command:

• rails new rails_on_win

The above command creates a new application I’m not going to get into that here you can find out more about the architecture of a rails app here.

Step 5 – Configuring the Database Connection (config\database.yml)

Next, change into the rails_on_win,directory or whatever name you used to create the application in step 4.  Open the config\database.yml file in your favorite editor.  Change the development section to be similar to this:

Change the DSN setting to whatever you called your DSN in step 0.  If you set up the default database in DSN or the DB user has one set you can get rid of the database setting.  if we were using a SQL Server account for authentication we would need to add username and password settings to the connection information.  Since we chose the integrated authentication we do not need to worry about it.  When you are done save and close the file.

Step 6 – Configuring and Installing the Gems in our App

Now that we have our database configuration complete the last thing we need to do before firing up the app is to update our Gemfile.  It can be found in the root directory of the app.  Open up your editor and load the Gemfile.  We need to add two entries:

• gem ‘ruby-odbc’
• gem ‘activerecord-sqlserver-adapter’

I added them after the gem ‘rails’, ‘3.0.1’ entry I’m not sure if order matters I’m just working under the assumption that it does.  After adding the two gems if you do not have Sqlite3 installed you will need to comment out the line: gem ‘sqlite3-ruby’.  Save and close the file.  We are now ready to fire up our app!

Step 7 – Starting the Application and Checking the Settings

In the application’s root directory run the command:

rails server 

It takes a little bit for the development server to start up but when it does you should see output similar to this:





Now we are ready to hit the web site.  Start up a browser and enter http://localhost:3000 you should see a page like this  (I’m just showing the top portion of the page):

If you see this page you are halfway there.  Next, click on the ‘About your application’s environment’ link.  You should see information in a yellow box that lists the configuration settings for your application.  The setting I want to point out is the database section towards the bottom of the page.  You should see:

If there is a setting that is incorrect or some other problem instead of the detailed setting description you will see an error message.  I ran into the one I mentioned in step 0 when I used the 64 bit DSN configuration applet. I received this error message when I misspelled the name of my DSN:

IM002 (0) [Microsoft][ODBC Driver Manager] Data source name not found and no
default driver specified

Step 8 – Create the Database

If you saw the page with all the settings information then you ready to get started!  At the same command prompt run:

rake db:create

After the successful completion of that command you should see a table called schema_migrations in your SQL Server database.

That’s It!

Now you are ready to start building rails apps in a Microsoft environment.

Sources

The two sources below in addition to the Rubyinstaller.org site  made this process extremely easy.  I had heard many people say it was hard to get rails going on windows.  I may run into that when I move out of the development phase but for now, it isn’t that bad. 

http://pragdevnotes.com/2010/02/18/ruby-rails-and-ms-sql-server

How to setup a 32-bit DSN on 64 bit Windows 7

Using IronRuby at Work: Part 1 – Using EF with IronRuby to retrieve data

• Part 1 – Using EF with IronRuby to retrieve data
• Part 2 – Using AssemblyBuilder and System.Reflection.Emit to create a .NET Assembly
• Part 3 – Creating the Domain class with CodeDom and IronRuby.

I will be giving a talk on this subject at this subject on November 6, 2010 at the Triangle .NET User Group’s RDU Code Camp

This is the first post in a three part series in which I will discuss how I used IronRuby to make my life easier.  In this post I’m going to discuss how IronRuby was used with Entity Framework 4 to retrieve data from a SQL Server database.

The Project – Use IronRuby to Produce a .NET Assembly

Our project consists of using 3 db tables to generate models and domain layer classes.  I chose to use IronRuby as the main language and to product an actual assembly that would contain the models and service classes. The goal of the project is to generate model and domain level code using 3 database tables.

The DB Tables

The three tables are elements, elements_to_views, and views.  The elements table holds all of the fields that will be needed to meet the requirements for this project.  The table’s main purposes is to satisfy a third-party requirement for documentation that means this table will always be up to date.  We will use this table as the main driver in the code generation process.  The views table is a way to group the elements into a set and we will use this to create the model classes.  Despite the name of this table these classes may or may not be used in the UI.  The elements_to_views table is used to relate the elements to their views which in the code means that the elements associated with a view will be a property in the view’s class. 

After setting up the EF entities I tried to interact with them directly from within the IronRuby script without success.  Since I’m new to IronRuby I’m not sure if it was something I was doing wrong or if it is a limitation, so instead of wasting a lot of time on figuring out I decided to create a Repository and ElementsService classes in C# to wrap the classes so I can get on with my IronRuby experimentation. (I’m not going to show the C# code here since I’m concentrating on IronRuby.  If you want to check out the code visit the the bitbucket project.)

Jumping into the IronRuby code…

Let me preface the rest of this post by saying I am a Ruby newb so please let me know if I”m breaking ruby coding practices or anything along those lines.  I’m learning it and loving it!  Any guidance would be appreciated.

After I finished the service class I was ready to start writing the IronRuby code.   My first class was a wrapper for the for the ElementsService C# class which I named….ElementsService.  Here is the code for the class:

What I’d like to discuss first are the two require lines at the top of the file.  The require ‘mscorlib’ brings in core Microsoft library.  The second is a reference to the assembly that contains the EF, Repository and ElementsService class.  The IRCDemo.Data assembly is in the same directory as the elements_service.rb file.Once those two requirements are loaded we have both assemblies at our disposal.  This means that any method that is available to you in C# is also available to you in IronRuby. 

Once we have the two assemblies loaded the script initializes the data context in the initialize method. The constructor in both Ruby and IronRuby classes.  The rest of the class just wraps calls to the C# version of the ElementsService class except for the get_properties method, which I will discuss in a later in the series.

Enough already, show me the data!

So now that we have the data access code in place lets write a script that gets all the data from the elements table.

Not much to this script but that’s the way I like it.  After loading the elements_service file the script instantiate an ElementsService object.  The next call is to the get_elements method which simply makes a call to the C# ElementsService.GetElements method.  When the results of the call are returned I loop through and print out the contents of the element record.  Once the data has been printed out the dispose method is called to ensure that the data context to ensure we clean up after ourselves.

To run the script above at the command prompt in the directory where the script exists enter:

ir blog.rb

will will produce output like this:

Summary

This is a good ending point for the first post in this series.  In this post we’ve laid out the foundation for the next two posts.  We’ve discussed how to interact with .NET assemblies from IronRuby.  In Part 2 we will take the .NET interoperability by using System.Reflection.Emit to create an assembly that contains the .NET model classes. 

You can download the code from here.

If You are Wondering Why I did this..

Why do this in IronRuby?  I’m in the process of teaching myself Ruby and Rails and the more I learn the more I like it.  So I’m looking for any excuse to introduce Ruby into our strictly MSFT environment.  Since this project is a proof of concept I thought it would be a nice starting point.

Addendum

I spent about five minutes attempting to get ActiveRecord to work with my SQL Server database without success that is why I went with the EF approach.  If anyone can point me to a resource that will help me get ActiveRecord up and running with SQL Server I’d appreciate it. Also, if there is anyone who has been successful in loading an EF generated model please let me know how you did  it.

It has been awhile…

Quite some time has gone by since I last posted an article on my blog.  Part of the reason is I was on vacation for a few weeks, another reason is school has started back up, and yet another reason is just pure laziness. 

In addition to those lame excuses I’ve been working on getting up to speed with Ruby.  I’ve dabbled with Ruby off and on for a little bit now.  I’ve used ruby to do a proof of concept here at work.  The POC worked out but was converted over to C# and MVC because our production environment set up.

In the next week I will start blogging about my experiences with IronRuby.  The first post will be around how I’m using IronRuby and Entity Framework 4 to access a MS SQL Server DB.  I’ll follow that up with either more details on that project or how I’m using RSpec and Cucumber to test our .NET based web applications.

MVCContrib – Portable Areas – Part 2 – Messaging

Update: After seeing a post on StackOverflow.com I have updated my sample code to work with the latest MvcContrib available through NuGet that supports MVC 2.

This is part two in a multi part series, the first entry is MVCContrib - Portable Areas - Part 1 - The Basics.  In that post we discussed how to setup and write a web site using a portable area (PA) that ran a Bing search.  In this entry we will discuss how to send messages between the PA and the consuming application.

Our application will be a ‘search’ application that has two different search engines (Engine One and Engine Two).  The PA will present the search forms and the search results.   The search strings will be sent to the consuming application to run the searches.  After the search is performed the results will be sent back to the PA to be displayed.

Requirements

The MVCContrib library which can be downloaded from MVCContrib download at CodePlex.  You will also need MVC 2 installed to be able to run the sample application.  I have built the example application using Visual Studio 2010.  I am planning on creating VS 2008 versions as well.

Lets Get Started!

The first thing we need to do is create the two projects, an MVC2 project for the consuming application, I named mine ConsumingApp,  The second project is a class library project for the PA, which I called ViewsPA.  Below are the required references:

1. ConsumingApp: 
   • MVCContrib.dll
   • And a reference to ViewsPA project
2. ViewsPA:
   • MVCContrib.dll
   • System.ComponentModel.DataAnnotations
   • System.Web
   • System.Web.Mvc
   • System.Web.Routing

ViewsPA

Since we talked about the layout of the code for the PAs in Part 1 - The Basics I won’t go into that here.  I will talk about the parts of the project that pertain to messaging.

The communication between the PA and consuming application is done via a synchronous message bus provided by the MVCContrib library.  Messages that are put on the bus by the PA must implement the MvcContrib.PortableAreas.ICommandMessage<T> interface where T is implements the ICommandResult interface.  The ICommandResult class is used by the consuming app to return the results to the PA.  Enough of this…here is the PA’s side of the messaging in C#.

In the SearchResult class the Success property is from the ICommandResult interface.  The remaining properties are there to illustrate the fact that you can have any number of properties and types of properties in the ICommandResult classes. 

The SearchMessage class is what is sent to the consuming application.  It implements the ICommandMessage<T> interface where the SearchResult class takes the place of T.  When you implement the interface the Result property must be the same type as T.  The Input property is a Query class which is a POCO and sends the search string to the consuming application.

The last bit of code I want to show is the controller’s action method.  Since both SearchOne and SearchTwo are doing the same thing but just setting which ‘search engine’ to use I have created a base controller class to make life easier.  The base class is here to process the user’s form submission and passes the query string to the consuming application.   

 

The first line is a normal POST handling method.  However once the model has been validated the controller starts the messaging process.

The first step is to create and load  a  SearchMessage object that will be sent consuming application.

Next the object is put on the bus and sent to the consuming application.

When the message is returned from the consuming application the method checks for success or failure of the processing.  If it fails do something to inform the user otherwise display the results of the ‘search’. 

That is all we need to do on the PA side.  We added a few classes and lines of code to enable our PA to communicate with the consuming application.

Consuming App

In our consuming application we need to create a class to handle the incoming messages from the PA and wire up global.asax to register our message handler with the messaging bus.

First we’ll take a look at what code we need to add to the global.asax file.  We need to add one line to the Application_Start method to receive the message being sent from the PA.  We added a line that calls the Bus.AddMessageHandler method to notify the bus that we have a message handler of type SearchHandler.  That class will handle all of our SearchMessage based messages from the PA. 

The SearchHandler class inherits from the MessageHandler<T> class where T is the message that this handler will ‘handle’. Our class will override the Handle method from MessageHandler.  This method is where we will do what we need to fulfill the PAs message.  Hopefully looking at the code will make it a little more understandable.

The Handle method has a single parameter of type T. In our make believe search engine case our method checks to see what engine the PA requested.  Instead of doing an actual search I am putting a canned message in the result object’s Message property that indicates which engine handled the request.  The Message and Results properties are used in the PA’s results view.  After this method finishes the PA’s controller method will pick up the results and call the appropriate view.

That is all there is to setting up messaging between the PA and the consuming app.

Lets Try It!

The home page is just about the same as it was in part 1 except that there are two tabs from the PA, Search One and Search Two.

 

 

 

 

 

 

 

 

Clicking on the Search One tab will take us to our new search form.  We have a second field here to distinguish between ‘search engines’.  When I submit my search string the PA sends the request to the consuming application which does the search and returns the results to the PA.  The results will display the name of the engine used and the results also indicate which engine returned the results.

Summary

Adding messaging to our search application only requires slight modification to our consuming application.  First we created a MessageHandler<T> based class that will respond to the PA’s message.  Next we added a line to the global.asax’s Application_Start method to register you handler class with the message bus. 

There was a little more work required on the PA side.  We had to alter the action method to send the message to the consuming application.  Once the consuming app is done the PA checks the results sent back the remaining portion of the action method is normal controller actions.

Why Would You Do This?!

Some of you may be wondering why you would do something like this with just the views on the PA.  Where I work we may be using this approach to create an application framework with pluggable modules for different sites.  The middle and data layers will be the same across all sites but certain locations will only use a few of the views.  This approach will help us create multiple sites without having to create a complete web site for each location.

What’s Next…

Part 3 of the series will either be on adding MEF to the picture or how to access a web.config file in the PA project. 

Downloads

• MVCContrib download at CodePlex.
• MVCContrib Portable Areas - Example App

Back from Tech-ED and I have OData Fever!

The title sums it up.  I am in the middle of an OData fit.  In my free time I am working on compiling baseball statistics from the Baseball Databank and Retrosheet sites to create one stats database that is available using OData.  I’m not sure how long it will take but I will have it up some time in the future.

What is OData?!

“The Open Data Protocol (OData) is a Web protocol for querying and updating data that provides a way to unlock your data and free it from silos that exist in applications today.” – taken from OData.org 

Serving data up this way frees us developers from some of the headache of cross platform data access (well at least those of us who primarily write for the windows platforms).  There are already quite a few client libraries out there to consume OData (PHP, Java, etc..).

I may have a few blog entries about OData and how to set it up on the server and client sides.  I still haven’t really thought it through all the way quite yet.

Why should I care about your fever?

If you are a baseball fan you will have access to historical data free of charge ready to use for whatever you can dream up.  In addition to the feed I plan on having a small app or two to demonstrate how to use the data and/or to query the data.

If you aren’t a baseball fan but are curious about OData then you may want to check back periodically to check my status.  I’m sure as I run into issues and/or figure things out I’ll blog about them.

Now if I could only find a few more hours to put into my day I could get this all done quickly.

MVCContrib – Portable Areas – Part 1

Update: After seeing a post on StackOverflow.com I have updated my sample code to work with the latest MvcContrib available through NuGet that supports MVC 2.

This is the first of a series of posts on how to use the MVCContib project’s portable areas functionality. Throughout this series we will be building out a ‘search’ application that utilizes portable areas (PAs) to provide the search functionality.  The consuming application is a relatively ‘dumb’ application that is used as a framework to serve up our PAs. 

In this entry I will discuss what libraries are necessary to use portable areas, how to configure your solution and projects and we will end the post with a basic web application runs a search using Bing.com.   The source code for this post can be found at MVCContrib Portable Areas - Example App at bitbucket.org

Requirements

The MVCContrib library which can be downloaded from MVCContrib download at CodePlex.  You will also need MVC 2 installed to be able to run the sample application.  I have built the example application using Visual Studio 2010 – Ultimate as my IDE but I will also include a vs2008 Professional version of the project.

Lets Get Started!

Start up Visual Studio and create a new ASP.NET MVC 2  Web Application to be the consuming application project, I named mine PortableAreas – Consuming App.  Next we need to create a Windows Library project which will be our portable area library, I named mine SearchPA project.   After creating the projects add the following references:

1. PortableAreas – Consuming App: 
   • MVCContrib.dll
   • And a reference to SearchPA project
2. SearchPA:
   • MVCContrib.dll
   • System.ComponentModel.DataAnnotations
   • System.Web
   • System.Web.Mvc
   • System.Web.Routing

The SearchPA Project

The layout of this project will be similar to a normal MVC project with a few changes.  The first change is the Bing folder.  Since we are starting off with Bing search all Bing related code will be under this folder.  The Controllers, Models, and Views folders are exactly the same as a normal MVC project.  The Messages folder will contain classes that are used to communicate with the consuming application.  We will talk more about Messages in the next post.  The other difference is the BingRegistration.cs file.

The BingRegistration class is used to register our PA with the consuming application and to inform the view engine about the presence of our routes and views.  It inherits from the class MvcContrib.PortableAreas.PortableAreasRegistration. From this class we override the RegisterArea method which has two parameters: AreaRegistrationContext and an IApplicationBus object.  The AreaRegistrationContext is used to map the Bing related routes.  The IApplicationBus is used for messaging between the PA and consuming app.  For now we are going to ignore the IApplicationBus but we will be used in the next post of the series.  After the routes are added we call a method to register the views with the view engine as you can see below.  The last task we need to do is override the AreaName property to set it to Bing.

There is one thing to keep in mind when you create your registration class, it must be at the base namespace of the controllers.  If the class in not in this namespace the portable area will not work.   

After the registration class has been completed the coding of the controllers and the models are the same.  In our case we will have an HTTP GET and HTTP POST version of an index method in the BingController class.  The contents of these methods are exactly the same as they would be for a ‘normal’ MVC app so I will let you read the code when you download the sample app.

The views in portable area are programmed in the same manner as normal views.  The only difference between the ‘normal’ MVC app’s views and the views in this project are that you must set them up as an embedded resource.   This allows the views to be packaged into the dll and the only thing the consuming application needs to do is add a reference to the SearchPA.dll.  In order to set up the view right-click on the view in the Solution Explorer and select Properties from the menu.  Select the Embedded Resource from the Build Action drop down.  Once that is done your view is ready. And with that we are done with the SearchPA project.

 

The Consuming MVC App

There are only a couple of additions needed to the MVC application in order to use our SearchPA portable area.  The first is configure the Global.asax file.

The code snippet above is from the global.asax file.  The only addition to this file for Part 1 of this series is the InputBuilder.BootStrap() line.  This line is necessary to initialize the embedded view engine used by the PAs.  In Part 2 we will add code to this method to allow communication bet the PA and consuming application.

The second modification is in the web.config file under the Area’s folder. The Areas/Web.Config file is needed to allow us to display the views that are part of the SearchPA project.  If you do not have this web.config file with the settings displayed below you will receive the error message below. 

The Areas/Web.config file is used to set the pages attribute validationRequest=”false”.  This allows the the embedded views to be served by the consuming application.

Let’s Try It!

Now that we have everything set up hit F5 and lets run the application. On the home page the Bing Search tab is from the SearchPA project.  Clicking on the ‘Bing Search’ tab is when the code that is part of the PA kicks in.

Here’s the search form.  This form is embedded into the SearchPA.dll.  When I do a search for ‘MVCContrib’ the results will be displayed by the results view in the SearchPA.  The consuming application does nothing but handle the routing.

And here is the results view…

Summary

Now we have a functional Bing search application using the MVCContrib Portable Area functionality.  We went over what you’ll need (MVCContrib download at CodePlex and ASP.NET MVC 2) and how to setup both projects for a self-contained portable area project.

What’s Next…

In the next article we will go over how PAs can communicate with the consuming application.

Downloads

• MVCContrib download at CodePlex.
• MVCContrib Portable Areas - Example App