Using SpecFlow to test my F# Baseball Stats Library

As part of my Ruby indoctrination I picked up a copy of The RSpec Book: Behaviour Driven Development with Rspec, Cucumber, and Friends (The Facets of Ruby Series) .  I’m about half way through the book and I find the Behavior Driven Development (BDD) process very comfortable.  Writing tests this way just ‘seems right’ and the process has already improved my Ruby code.  However, during my day job I write C# code so I started looking around to see what BDD options are available for .NET.  That’s when I found SpecFlow, which plays the role of Cucumber in the Ruby world.  So to get up to speed with SpecFlow I’ve decided to use it to help me test my obp function which I wrote in Project Chadwick #2–Top 5 SF Giants OBP (F# Version) as I build out my baseball stats library.  Before I get started I will describe the setup.

The Setup

Step 0. Create F# Library Project

On the File Menu in Visual Studio Select New Project.  In the New Project dialog, open the Other Language option and click on Visual F# then Select F# Library.

I named this project BaseballStats.  Remove the Module.fs and Script.fsx files, I’ll add my own *.fs file when the time comes and we won’t need the Script.fsx file. That’s it for the F# project until it is time to create test test project. 

Step 1. Create the Test Project

Since I am using MSTest for the testing I am using a normal C# test project.  I named mine BaseballStats.AccetpanceTests.  Once the project is created I need to add the following references:

1. FSharp.Core assembly
2. F# library project you created in Step 0, in my case the BaseballStats project.

Delete the test class that was automatically added to the test project, I wont be using it.  In real life I’d create a unit test project also lets just pretend we did that here.  

Now that we have the test project created its time to install SpecFlow

Step 3. Install SpecFlow

If you already have SpecFlow installed then you can jump down to the NuGet section, otherwise go ahead and grab SpecFlow installer from here.  I downloaded and ran the installer so that I had the file templates available in the ‘Add New Item’ dialog.  After the install I ran the command below from the NuGet Console to add the necessary DLLS to the test project.

install-package –Id SpecFlow –Project BaseballStats.AcceptanceTests 

SpecFlow uses NUnit as its test runner by default but it can be configured to use MSTest.  Since I’m using MSTest I need to update the app.config file so it looks like this:

<specFlow>
    <!-- Possible values include NUnit (default), MsTest, xUnit -->
    <unitTestProvider name="MsTest" />
  </specFlow>

The setup is complete.  Its time to get on with the testing!

The Testing

Step 4.  Create a Feature

To create a SpecFlow feature file right click on the BaseballStats.AcceptanceTests project and select ‘Add…’ > ‘New Item…’ And Select SpecFlow feature file.  Name it CalculatingObp.feature. 

Any file with the .feature suffix will also have a code behind file that SpecFlow will use to call the step definition methods.  The step definitions are what is run to perform the tests. I will define the steps after I have finished describing the feature .  When a new feature file is created you will see the following:

Feature: Addition
In order to avoid silly mistakes
As a math idiot
I want to be told the sum of two numbers

@mytag
Scenario: Add two numbers
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press add
Then the result should be 120 on the screen

Obviously this isn’t the feature I want to describe but lets take a minute to discuss it.  What the file does is describe the feature we are working in an almost plain English style.  The text under the Feature line is a narrative to help remind me what I want the the feature to do.  It has no real bearing on the code that we will use to test the feature. 

The Scenario section does have an impact on the test’s code. The Given, And, When and Then statements will be used in the step definition file and they will drive the test. 

Here’s what our feature for calculating the OBP looks like:

Feature: Calculating OBP
In order to determine the effectiveness of a batter
As a baseball fan
I want to be able to calculate a player's On Base Percentage

Scenario: Calculate a Season's On Base Percentage (OBP)
Given A batter had "389" ABs, "104" Hits, "56" BBs, "0" HBPs,
and "4" SFs
When I run the calculation 
Then I should see the result "0.356"

The feature is used to describe how I am going to calculate a batter’s seasonal OBP.  When I save the feature file a code behind file is created that contains code that SpecFlow will use to find the step definition. 

Step 5. Create the Steps

Now that I have the feature description in place I’m ready to test.  Lets run the SpecFlow test and see what happens.  To run the test make sure the feature file is the selected tab in VS and click the ‘Run Test in Current Context’ button.  When you run the test the results will say ‘Inconclusive’.  View the test run details and you will see a statement that says there were no matching steps found for …. which maps to the first line of the scenario.  A little further down in the ‘Standard Console Output’ section you will see that SpecFlow has provided us with boiler plate code for the step definitions that looks like:

Given A batter had "389" ABs, "104" Hits, "56" BBs, "0" HBPs, and "4" SFs
-> No matching step definition found for the step. Use the following code to create one:

[Binding]
public class StepDefinitions
{
[Given(@"A batter had ""389"" ABs, ""104"" Hits,   
""56"" BBs, ""0"" HBPs, and ""4"" 
SFs")]
public void GivenABatterHad389ABs104Hits56BBs0HBPsAnd4SFs()
{
ScenarioContext.Current.Pending();
}
}

When I run the calculation I should see
-> No matching step definition found for the step. Use the following code to create one:
[Binding]
public class StepDefinitions
{
[When(@"I run the calculation I should see")]
public void WhenIRunTheCalculationIShouldSee()
{
ScenarioContext.Current.Pending();
}
}

Then I should see the result "0.356"
-> No matching step definition found for the step. Use the following code to create one:
[Binding]
public class StepDefinitions
{
[Then(@"I should see the result ""0\.356""")]
public void ThenIShouldSeeTheResult0_356()
{
ScenarioContext.Current.Pending();
}
}

It is time to add a step definition file to our testing project.  Right click on the AcceptanceTests project and select Add New Item Select the SpecFlow Step Definition option and give it the name ObpStepDefinitions.

Remove the template code that is inserted into the ObpStepDefinitions test and replace it with the boiler plate methods, from the ‘Standard Output Console’ area of the test results, that were generated when when ran the SpecFlow test. My step definitions should now look like this:

Run the test again and this time the test should report:

Assert.Inconclusive failed. One or more step definitions are not implemented yet.
ObpSteps.GivenABatterHad389ABs104Hits56BBs0HBPsAnd4SFs()

This is a good sign!  What it means is that SpecFlow now sees the steps and attempts to execute them but since the only code in the first method is the ScenarioContext.Current.Pending method call and it halts execution there since this is the first step.  Now its time I put some actual code into the method bodies.  I am going to start with the Given step. I’m going to use the numeric values in the Given statement as inputs for my test.  How can I do that?  With SpecFlow I can use regexes to grab the numeric values from the Given attribute so we can use them to run the OBP calculation.   The values grabbed by the regexes are then passed to the step method via parameters.  The values will be converted to the specified data type in the method signature by SpecFlow.  My updated step looks like this:

This step is responsible for retrieving and storing the input values that will be used to calculate the OBP. Now when I run the test I see:

Assert.Inconclusive failed. One or more step definitions are not implemented yet.
ObpSteps.WhenIRunTheCalculationIShouldSee()

Again, it is a good sign.  It actually executed the first step and is now trying to run the When step, but it encounters the Pending method again.  This is the step where will do the OBP calculation. Since we do not have the Baseball.obp function in the F# code, we are adding code to the step that 'We wish we had', a phrase the author users repeatedly in the RSpec book. Here is the update step definition.

Since the Baseball.obp function doesn't exist yet we will not be able to build the project. So I am going to switch to the F# BaseballStats project and write just enough code to allow us to build and run the test. First we create a BaseballStats.fsi file followed by a BaseballStats.fs file. In F# projects a file’s order of appearance matters in the build process.  Make sure that the fsi file appears before the fs file in the project’s listing.  To move a file up or down right click on the file you wish to move and choose the appropriate movement direction.

The BaseballStats.fsi file is a signature file, you can think of it like a C/C++ header file. It describes the functions that are available in the BaseballStats.fs file. The val obp line is describing the function's signature. There will be 5 float parameters and it will return a float value.

The BaseballStats.fs file is where the function is implemented. In the real world we’d write just enough code to allow us to run the test again and when it failed we’d drop into unit testing or a RSpec .NET equivalent until the obp function was fully functional.  Then we’d come back to SpecFlow, run the test and get green.  In order to keep this post as brief as possible I’m not going to illustrate the process here. I have added the entire obp function but pretend we went the through process I just discussed.  Once I’ve added the obp function to the fs file build the solution and run the test.   It still comes up as Inconclusive. This time it is due to the last step not being defined.

The final step in my test is assert that the calculated OBP equals the expected value.  Again I’m using regex to grab the expected value which will compared against a rounded off version of the results from the Batting.obp call. Once we have green we know that the feature is working for this set of test data.

Here is what the detailed view of the test run should look like:

That’s it, we have green!  The OBP function performed as we had expected.  Obviously we haven’t fully tested it but we now know that the function works with valid inputs.  So what happens when we provide negative values or values that make the denominator zero? SpecFlow has a way that will allow me to use this single scenario to test all the possible permutations I can think of without writing additional scenarios.  I’m going to save that topic for a later post.

My Thoughts on SpecFlow

SpecFlow gives .NET developers a way to get BDD into our projects.  In the beginning using SpecFlow doesn’t seam to flow as smoothly as Cucumber and RSpec in the ruby world.  This may be due to the fact that I haven’t used SpecFlow enough or could be due to the C# and Ruby differences.  Overall, I like the BDD style of development that SpecFlow brings to the .NET world.  BDD seems to fit better to my way of thinking.  I am going to continue to use SpecFlow in my side projects and will work on incorporating it into my ‘day job’ environment.

Resources

You can download the source here

SpecFlow: project web site

TekPub’s free video on SpecFlow

F#: fsharp.net