Tuesday, January 25, 2011

Using SpecFlow to test my F# Baseball Stats Library

As part of my Ruby indoctrination I picked up a copy of The RSpec Book: Behaviour Driven Development with Rspec, Cucumber, and Friends (The Facets of Ruby Series) .  I’m about half way through the book and I find the Behavior Driven Development (BDD) process very comfortable.  Writing tests this way just ‘seems right’ and the process has already improved my Ruby code.  However, during my day job I write C# code so I started looking around to see what BDD options are available for .NET.  That’s when I found SpecFlow, which plays the role of Cucumber in the Ruby world.  So to get up to speed with SpecFlow I’ve decided to use it to help me test my obp function which I wrote in Project Chadwick #2–Top 5 SF Giants OBP (F# Version) as I build out my baseball stats library.  Before I get started I will describe the setup.

The Setup

Step 0. Create F# Library Project

On the File Menu in Visual Studio Select New Project.  In the New Project dialog, open the Other Language option and click on Visual F# then Select F# Library.


I named this project BaseballStats.  Remove the Module.fs and Script.fsx files, I’ll add my own *.fs file when the time comes and we won’t need the Script.fsx file. That’s it for the F# project until it is time to create test test project. 

Step 1. Create the Test Project

Since I am using MSTest for the testing I am using a normal C# test project.  I named mine BaseballStats.AccetpanceTests.  Once the project is created I need to add the following references:

  1. FSharp.Core assembly
  2. F# library project you created in Step 0, in my case the BaseballStats project.

Delete the test class that was automatically added to the test project, I wont be using it.  In real life I’d create a unit test project also lets just pretend we did that here.  

Now that we have the test project created its time to install SpecFlow

Step 3. Install SpecFlow

If you already have SpecFlow installed then you can jump down to the NuGet section, otherwise go ahead and grab SpecFlow installer from here.  I downloaded and ran the installer so that I had the file templates available in the ‘Add New Item’ dialog.  After the install I ran the command below from the NuGet Console to add the necessary DLLS to the test project.

install-package –Id SpecFlow –Project BaseballStats.AcceptanceTests 

SpecFlow uses NUnit as its test runner by default but it can be configured to use MSTest.  Since I’m using MSTest I need to update the app.config file so it looks like this:

    <!-- Possible values include NUnit (default), MsTest, xUnit -->
    <unitTestProvider name="MsTest" />

The setup is complete.  Its time to get on with the testing!

The Testing

Step 4.  Create a Feature

To create a SpecFlow feature file right click on the BaseballStats.AcceptanceTests project and select ‘Add…’ > ‘New Item…’ And Select SpecFlow feature file.  Name it CalculatingObp.feature. 


Any file with the .feature suffix will also have a code behind file that SpecFlow will use to call the step definition methods.  The step definitions are what is run to perform the tests. I will define the steps after I have finished describing the feature .  When a new feature file is created you will see the following:

Feature: Addition
In order to avoid silly mistakes
As a math idiot
I want to be told the sum of two numbers

Scenario: Add two numbers
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press add
Then the result should be 120 on the screen

Obviously this isn’t the feature I want to describe but lets take a minute to discuss it.  What the file does is describe the feature we are working in an almost plain English style.  The text under the Feature line is a narrative to help remind me what I want the the feature to do.  It has no real bearing on the code that we will use to test the feature. 

The Scenario section does have an impact on the test’s code. The Given, And, When and Then statements will be used in the step definition file and they will drive the test. 

Here’s what our feature for calculating the OBP looks like:

Feature: Calculating OBP
In order to determine the effectiveness of a batter
As a baseball fan
I want to be able to calculate a player's On Base Percentage

Scenario: Calculate a Season's On Base Percentage (OBP)
Given A batter had "389" ABs, "104" Hits, "56" BBs, "0" HBPs,
and "4" SFs
When I run the calculation 
Then I should see the result "0.356"

The feature is used to describe how I am going to calculate a batter’s seasonal OBP.  When I save the feature file a code behind file is created that contains code that SpecFlow will use to find the step definition. 

Step 5. Create the Steps

Now that I have the feature description in place I’m ready to test.  Lets run the SpecFlow test and see what happens.  To run the test make sure the feature file is the selected tab in VS and click the ‘Run Test in Current Context’ button.  When you run the test the results will say ‘Inconclusive’.  View the test run details and you will see a statement that says there were no matching steps found for …. which maps to the first line of the scenario.  A little further down in the ‘Standard Console Output’ section you will see that SpecFlow has provided us with boiler plate code for the step definitions that looks like:

Given A batter had "389" ABs, "104" Hits, "56" BBs, "0" HBPs, and "4" SFs
-> No matching step definition found for the step. Use the following code to create one:

public class StepDefinitions
[Given(@"A batter had ""389"" ABs, ""104"" Hits,   
""56"" BBs, ""0"" HBPs, and ""4"" 
public void GivenABatterHad389ABs104Hits56BBs0HBPsAnd4SFs()

When I run the calculation I should see
-> No matching step definition found for the step. Use the following code to create one:
public class StepDefinitions
[When(@"I run the calculation I should see")]
public void WhenIRunTheCalculationIShouldSee()

Then I should see the result "0.356"
-> No matching step definition found for the step. Use the following code to create one:
public class StepDefinitions
[Then(@"I should see the result ""0\.356""")]
public void ThenIShouldSeeTheResult0_356()

It is time to add a step definition file to our testing project.  Right click on the AcceptanceTests project and select Add New Item Select the SpecFlow Step Definition option and give it the name ObpStepDefinitions.


Remove the template code that is inserted into the ObpStepDefinitions test and replace it with the boiler plate methods, from the ‘Standard Output Console’ area of the test results, that were generated when when ran the SpecFlow test. My step definitions should now look like this:

Run the test again and this time the test should report:

Assert.Inconclusive failed. One or more step definitions are not implemented yet.

This is a good sign!  What it means is that SpecFlow now sees the steps and attempts to execute them but since the only code in the first method is the ScenarioContext.Current.Pending method call and it halts execution there since this is the first step.  Now its time I put some actual code into the method bodies.  I am going to start with the Given step. I’m going to use the numeric values in the Given statement as inputs for my test.  How can I do that?  With SpecFlow I can use regexes to grab the numeric values from the Given attribute so we can use them to run the OBP calculation.   The values grabbed by the regexes are then passed to the step method via parameters.  The values will be converted to the specified data type in the method signature by SpecFlow.  My updated step looks like this:

This step is responsible for retrieving and storing the input values that will be used to calculate the OBP. Now when I run the test I see:

Assert.Inconclusive failed. One or more step definitions are not implemented yet.

Again, it is a good sign.  It actually executed the first step and is now trying to run the When step, but it encounters the Pending method again.  This is the step where will do the OBP calculation. Since we do not have the Baseball.obp function in the F# code, we are adding code to the step that 'We wish we had', a phrase the author users repeatedly in the RSpec book. Here is the update step definition.

Since the Baseball.obp function doesn't exist yet we will not be able to build the project. So I am going to switch to the F# BaseballStats project and write just enough code to allow us to build and run the test. First we create a BaseballStats.fsi file followed by a BaseballStats.fs file. In F# projects a file’s order of appearance matters in the build process.  Make sure that the fsi file appears before the fs file in the project’s listing.  To move a file up or down right click on the file you wish to move and choose the appropriate movement direction.


The BaseballStats.fsi file is a signature file, you can think of it like a C/C++ header file. It describes the functions that are available in the BaseballStats.fs file. The val obp line is describing the function's signature. There will be 5 float parameters and it will return a float value.

The BaseballStats.fs file is where the function is implemented. In the real world we’d write just enough code to allow us to run the test again and when it failed we’d drop into unit testing or a RSpec .NET equivalent until the obp function was fully functional.  Then we’d come back to SpecFlow, run the test and get green.  In order to keep this post as brief as possible I’m not going to illustrate the process here. I have added the entire obp function but pretend we went the through process I just discussed.  Once I’ve added the obp function to the fs file build the solution and run the test.   It still comes up as Inconclusive. This time it is due to the last step not being defined.

The final step in my test is assert that the calculated OBP equals the expected value.  Again I’m using regex to grab the expected value which will compared against a rounded off version of the results from the Batting.obp call. Once we have green we know that the feature is working for this set of test data.

Here is what the detailed view of the test run should look like:


That’s it, we have green!  The OBP function performed as we had expected.  Obviously we haven’t fully tested it but we now know that the function works with valid inputs.  So what happens when we provide negative values or values that make the denominator zero? SpecFlow has a way that will allow me to use this single scenario to test all the possible permutations I can think of without writing additional scenarios.  I’m going to save that topic for a later post.

My Thoughts on SpecFlow

SpecFlow gives .NET developers a way to get BDD into our projects.  In the beginning using SpecFlow doesn’t seam to flow as smoothly as Cucumber and RSpec in the ruby world.  This may be due to the fact that I haven’t used SpecFlow enough or could be due to the C# and Ruby differences.  Overall, I like the BDD style of development that SpecFlow brings to the .NET world.  BDD seems to fit better to my way of thinking.  I am going to continue to use SpecFlow in my side projects and will work on incorporating it into my ‘day job’ environment.


You can download the source here

SpecFlow: project web site

TekPub’s free video on SpecFlow

F#: fsharp.net

Thursday, January 20, 2011

Project Chadwick Update

I had hoped to crank out Project Chadwick posts about once a week but as you can tell that hasn’t happened.  One of the reasons is I was a little too ambitious writing them in four different languages especially since I’m learning the languages as I go (except for ruby).  Plus I’ve been working on a few side projects that have taken more time than I had anticipated.  So, what I’ve decided to do is to continue with the series using only two languages F# and Clojure. 

Why these two languages?

Basically, I’ve become very curious about functional languages so why not use this series as a starting point.  One of the reasons I’m staying with F# is it as a CLR based language which means that I may be able use it in future projects at my day job. I’m replacing Erlang with Clojure simply because I went to a Raleigh.rb, the local ruby meetup group, meeting where they had talk on Clojure.  The speaker made Clojure sound fun so I thought I’d give it a try.  I will be using the port of Clojure that runs on the CLR simply for ease of development for me.

Up Next…

I will be adding a few more problems to the Problems Page in addition to going back to the beginning and writing solutions in Clojure.  I’m looking forward to delving into the world of Clojure and functional programming in general.

Tuesday, January 11, 2011

Project Chadwick #2–Top 5 SF Giants OBP (Ruby version)

Before I get started on this post if you aren't familiar with Project Chadwick here's a quick overview. The data for this problem can be downloaded from here

The Problem

The On Base Percentage or OBP is the percentage of time the player reaches base. It is calculated using the following formula:
OBP = (H + BB + HBP) / (AB + BB + HBP + SF)

H   = Hits
BB  = Walks
HBP = Hit By Pitch
AB  = At Bats
SF  = Sacrifice Flies

The batter should have had at least 200 at bats to be eligible. The results should be printed out with in the following format: players name, season, and OBP.

The Solution

New Ruby Concepts

Ruby Class

The first ‘new’ thing you see in this script is a class. There are three items I'd like to discuss about this class:  attr_accessor, initialize and to_s methods.

The attr_accessor is a method that is used to indicate which instance variables, also known as attributes, are available outside of the class. All of the variables declared in the attr_accessor call can be read and updated from outside of the class. You can also set up attributes that are read-only by calling attr_reader. To set up a write-only attributes by calling the attr_writer. When attributes are used within a class they are prefaced with @ sign.

Another feature of a Ruby class is the initialize method which is the class’s constructor. In this class there isn’t much going on here other than taking the stats and putting them in a variable that makes it easier to read the code.

Well, there is a little more going on then a simple assignment. In the stats fields the ||= says that if the value on the left is nil then assign 0.0 to it. The value is then being converted to float so we can do division.

The last method I want to discuss is the to_s method. This method is overriding the base class’s to_s which is the a to string class. In this method I’m just formatting the output of the class so my loop is a little cleaner.


One of my favorite things with Ruby is how arrays can be manipulated and used. The first example I’m adding the newly created Batter object to the end of the all_obps array. In the second line I'm sorting the array by the obp value. What I'd like to point out here is the ! at the end of sort. The ! indicates that the sorting will be done in place, in other words it changes the value of the array I’m sorting. If I left the ! off the sort would create a new array that was sorted leaving me with the original array AND a new, sorted array. Finally, the reverse! line has the [1,5] added to it. Which means I want the first through the fifth elements. In our case that is how I grab the top five OBP for the SF Giants.

An Overview of My Solution

In this script I’ve opted to write code that was easier to read instead of the fewest lines that’s why I created the Batter class. This way we can access the stats needed to calculate the OBP in a more straight forward way. Since I’ve already pointed out some of the nuances of Ruby classes and the calc_obp method being straight forward I’m not going to going to dissect the class any further here.

What I will talk about how the script reads and processes the data.  The first line in the snippet reads all lines of the file and then loops over it. I then create the Batter object and check to see if the batter had at least 200 at bats. If not I move on to the next line. If the batter had at least 200 at bats I calculate the OBP and store the Batter object in the all_obps array. The rest of the script sorts, reverses and prints the top five OBP for the San Francisco Giants.

That’s it, nothing to the Ruby script when you compare it to the F# script.  This may be due to the fact that I’ve worked with Ruby more than I have F# but I believe Ruby is just a little more concise than the F#.  I may change my mind after writing more F# code but for now that’s the way it seems to me.

I really enjoy writing Ruby code, it allows me to concentrate on what I need to do more than 'how do I do this in Ruby'. I guess what I'm trying to say it feels more natural than other languages such as C#.

Up Next...

I'm still working on getting more problems up for those of you who are interested in continuing on the baseball stats path. I'll write the Erlang version next and then wrap up the second problem with the Objective-C code.

Thanks for stopping by.