My Generator in the ‘Real World’–Generating Code from a JSON source

A while back I wrote a blog about using IronRuby to generate assemblies.  Back then the project had a pretty narrow goal which was to read information from a group of database tables and generate C# classes to model the data describe in the three tables.  It works nicely for what I needed for my day job.  While I was working on the original project I thought it would be nice to add more input sources like text files, URLs, etc.  In addition to multiple data sources I wanted to add the ability to produce code for other languages such as Ruby.  Given the time constraints I was working in I put those additions off.  Recently I had some time free up (yeah, right!) so I created a second project that is aimed at a wider field of use, Linux, Mac as well as windows.  I have added Ruby as an output language plus a new URL source.  .

In this blog will illustrate how a URL source that returns JSON can be used to create C# code.  The generated code will be responsible for retrieving the data and converting it from JSON to C# classes.  Next, I will be incorporated into a MVC project that I will use to display the standings from the given season.  The goal here is to show how quick you can use the code to display on the page.

Step 0.  Create new ASP.NET MVC 3 Project

Go through the usual steps of creating a new project in Visual Studio. Once you have created the project fire up the Nuget console and run:

Install-Package Newtonsoft.Json

The Newtonsoft.Json package is used by the service class we will generate in the next step.  The library will convert the JSON data and loads the C# class.

If you don’t have Nuget you can go the nuget.org site or install it through the Visual Studio Extension Manager.  It is an easy way to install third-party libraries to your project.  It feels very much like installing a gem in ruby. 

Ok, we have the project the way we need it now its time to generate the classes.

Step 1. Generating the Classes

I have created a batch file to run the code generation script so I don’t have to type out all the options.  The more I type the more mistakes I make.  Anyway here is the batch file.

The -i and -url parameters are used together to tell the generate that we are using URL as an input source and the –url passes the actual URL to use. The -l c_sharp and -mc switches are used to pass the desired output language and the name of the primary model class. The -mod and -sod options are used to tell the generator where to write the model and service files respectively. 

Next step is to run the script which takes all of a second and now I have my classes. 

A total of seven classes were generated to model the JSON (View the JSON here) that was returned.  The Standings.cs file is the primary model, meaning it is the topmost class in the source.  All other classes that end with Model were discovered in the input parsing process.  The all represent a property of the primary model or one of its supporting models.

The StandingsService.cs class will be used in our MVC project to retrieve the data and populate the Standings class.  We will use that class to display the data in a view.

Now that I have my JSON based classes it is time to start creating the web front end.

Step 2.  Incorporating the StandingsService Class

Since I chose to create an empty MVC project the first thing I’m going to do is create a StandingsController class without the CRUD methods.  I also created an index view that we will use to display standings.  Here is the code that I’ve added to the controller’s index method to retrieve the standings.

As you can see there is much to this method.  I am instantiating the StandingsService class and calling the Get method which returns an IEnumerable.  Since I know there will only be one object in the list I’m grabbing that and sending it to the view.  If this was actual production code I’d have a few more lines here to validate the year parameter, have the URL in the web.config file and check for nulls before I sent the data off to the view. Since its demo code I’m keeping it simple.

The same goes with the view, demo simple. I sort the values by league, division and ranking.

Nothing fancy here but going from nothing to displaying data in less than 10 minutes is a good way to start any project.  The generator takes away the tedious part of data access, creating the models and service classes and lets me get down to the business layer of the application. 

 

 

Summary

I started this project off with a URL to a web site that returns MLB standings in JSON format and I wanted to be able to display the results in my web site.  Using my generator project I was able to create the model classes and a service class needed to manipulate the data in C# in less than five seconds.  After the classes were generated it took me somewhere between 5 to 8 minutes to get a web page up and running that took the data from the URL and displayed it in my MVC view.

The generator project very young and hasn’t had many real world tests.  In the near future I will be running it through more rigorous testing.  Look for more blog posts on and around the generator project.

The MVC project can be downloaded from here

The Generator code  (Remember, this is very young/green code)

Part 3 – Creating the Domain class with CodeDom and IronRuby

• Part 1 – Using EF with IronRuby to retrieve data
• Part 2 – Using AssemblyBuilder and System.Reflection.Emit to create a .NET Assembly
• Part 3 – Creating the Domain class with CodeDom and IronRuby.

This is the final post in a three part series in which I discuss how I used IronRuby to generate data access code.  In this post I’m going to discuss how IronRuby was used to generate a C# domain class for each model.

All source files can be downloaded from here

Generating the Domain/Service Layer Code

Now that we have models to truck the data from the DAL to the presentation layer we need the code to move the data between the two layers.  In this version of the project I used the CodeDom to generate the C# code.  This will change as I move this project more into our production environment.  I plan on moving towards a templating engine such as Ruby’s ERB or ASP.NET MVC 3 Razor view engine.  In the beginning of the project I had intended to write this layer using the Emit approach but it found it to be overkill and at this layer the chances are better that we will need to modify the generated code which is not possible if we go down the Emit path.  With that said lets dive into the CodeDom approach. 

In this post I will walk you through how I created the class, added a private field, the constructor

System.CodeDom – The Setup

In order generate the code we must first create a CodeCompileUnit object.  Think of it as a container for the code tree we are about to create.  Next we CodeNamespace object passing in the namespace that will contain the class we are creating. The last bit of setup we’ll do is to import any namespaces our class will need using the CodeNamespaceImport object.Here’s what the setup code looks like:

Creating the Domain Class

Now we can actually create our class type using the CodeTypeDeclaration class passing in the name of the type we are creating.  This call creates a type but we still need to indicate we are creating a class by setting the CodeTypeDeclaration.IsClass property to true.  We’ll also make the class public by setting the TypeAttributes property to TypeAttributes.Public.  After creating the class type we’ll add it to the namespace by passing the type to the @nspace_holder.Imports.Add method.  The code for creating the class type is below.

Adding a Field

Our class will need a field to store the reference to the repository it will use. The first step is to create a CodeMemberField object and set the attributes to private, give it the name _repo and set the type to be IRepository.  Next we’ll add it to our class by adding it to the Members list.

Adding the Constructor

Once we have the field to hold the repository we need to set it to something.  The constructor will have one parameter IRepository repo.  The body of the constructor will have a check to ensure that the repo parameter is not null.  If it is null it will throw an ArgumentNullException.

Creating the constructor object is as simple as create a new CodeConstructor object.  We make the constructor public by setting its attributes to MemberAttributes.Public.  Next, the IRepository parameter is added to the Parameters list by creating a CodeParameterDeclarationExpression object passing in the type and name of the parameter. 

Once we have created the parameter we need to grab a reference to it so we can use it to set the class’s _repo field to the value of the parameter. We pass the parameter and field references to the CodeAssignmentStatement constructor.  This object will be used as the true statement in the if/else code which is created when we call the create_if_else_statement method.

As you can see this method is pretty straight forward.  You pass in a condition to test, the statements to execute if the condition is true or false.  It returns the statement that we will add to the constructor object’s Statements list. The last step is to add the constructor to the class's Members list.

Creating the Get Methods

The domain class has two Get methods. The first one returns an IQueryable that when executed would return all records. The second will return the record that matches the Id parameter that is passed in.

All methods we create are started by calling the basic_method.  This method instantiates a CodeMemberMethod object, setting the Attributes to be public method, the name of the method and the return type.

The first get method

This method will return an IQueryable.  In order to set that up we create a CodeTypeReference object passing in IQueryable<T> where T is the EF model type to the constructor. Our next step is to add the return statement to the method’s body.  This is done by creating a CodeMethodReturnStatement passing the results of the create_repo_method_call method to it’s constructor.

The create_repo_method_call is a way to create calls to the repository class.  It creates a CodeMethodInvokeExpress object that represents the method we will call in our method.  It takes 3 parameters.  The first is a CodeTypeReferenceExpression which in our case is the _repo field. The second parameter is the name of the method we are going to call, in this case its Get. The final parameter takes an array of parameters that the called method receives.  If there are no parameters it is an empty array.

After the create_repo_method_call returns and the results of the call are stored in the method’s Statements property the method is added to the class’s Members list.

The Second get method

The second Get method takes an Id parameter which is used to find the single record that matches it. After the method’s CodeMemberMethod object is created the first thing we do is add the Id parameter following the same process we did with the constructor and create a reference to it.  Next, a MethodInvokeExpression object is created to make the call to the repository’s Get method.  In addition to this call there will be two other method calls chained to it.  The first is a LINQ Where method that takes a CodeSnippetExpression object as its parameter.  Finally an FirstOrDefault method is added to the chain.  Here is the code for the add_get_methods.

The Complete Domain/Service Generated Through this Process

Summary

Now we have the domain/service layer classes to go with the models we created in Part 2. The C# classes have methods to Get, Add, Update, Delete records that map to the models we’ve created.  These classes also have methods to map between the EF models and our models.  Generating these two layers of code and adding them to our MVC projects gives us the potential to have basic application up and running quickly.

What’s next with this project?  On the System.Emit portion of the project I will be adding the ability to store data from one model class into multiple tables, add attributes to the properties, and a way to generate models from non-database data sources such as flat files, URLs and HL7 messages.  I will continue to generate the models using the System.Emit process.  I will be changing my approach on how the source code is generated to use either Ruby on Rails’ ERB view engine or the new Razor view engine in ASP.NET MVC.  I believe this approach will make it easier for us to make changes to the process and makes it easier to maintain in the long run. I will be adding the ability to generate unit tests, controller class boiler plates and perhaps HTML views as well.

I enjoyed working on this series and I hope you enjoyed it!