Friday, August 31, 2007

Moving to a new blog

I've decided to create my own blog site. I'ts called justnbusiness and that is where I'll be posting from now on. I did it for a couple reasons, one was because I was tired of not being able to upload zip files to blogger. It's great that they let you do images but I really want to be able to do zip files since I regularly want to post code snippets along with the blog. Another reason is because I wanted to have a visible working example of NBusiness in action (even if it is a simple one). I wanted to 'eat my own dog food', if you will.

So if you have subscribed to this RSS feed (why do I hear crickets chirping?) feel free to delete it and head over to http://www.justnbusiness.com and subscribe there instead! This will likely be my last post to this blog.

Monday, August 27, 2007

RC2 Available

I just uploaded a new release candidate this weekend, cleverly dubbed 'RC2'. This is the first v2 build that will actually install as visual studio plugin and work correctly without requiring the Visual Studio SDK. Be sure to read the README.txt that comes with the build. There were two tricks with fixing the problems with that last RC.
  1. If you don't have the SDK you at least need the ProjectAggregator2.msi, which is a redistributable that comes with the SDK.
  2. I had to get the version information in the registry keys of the installer up to date with the new assemblies. This version has a new PLK that I got off of the Microsoft VSIP website as well. I had been adjusting it in code but forgot the installer required some manual updates.

I plan on upgrading the installer project to a WIX projects as soon as possible. I don't like that the project aggregator redist. isn't part of the automatic install process and it's practically impossible to figure out how to do that with the old school install process. I'm not really looking forward to redoing the installer but I want a double click-next-next-next install process and anything less is unworthy.

There are a few known issues with this release, most of them are listed on the Issues List on codeplex, for example the oh so annoying error message about not being able to get references for your assembly despite the fact that your project will successfully build. I've never seen an error message that lets you build still before. But this is a known bug in the VS SDK, hopefully it'll be fixed on the next release. I guess I'll have to try to get this thing working with VS2008 sometimes soon as well.

Anyway download RC2 and let me know what you think! Someone start up another good thread on the forums!

Tuesday, July 31, 2007

NBusiness v2.0 RC1

So it's here at last! This is the not quite officially released version of NBusiness 2.0. Here is the link:


For those of you who have tried out my Alpha/Beta versions of 2.0 in the past I'll say that this is a much easier install to get working. This is the real-deal, no experimental mode BS to have to deal with. Full integration baby!

I should say that you should expect some bugs for now but it should be "mostly" working. Meaning, simply firing up a new project should just build right out of the box.

Most core features should be working just as you'd expect but I haven't tested out the VB side of things thoroughly enough yet so don't expect much from there. If you test it out and find some bugs please enter an issue in the issue list! I'd really appreciate it.

After doing some thinking about the time spent getting to this point I realized that I haven’t been recording hours at all and it doesn’t seem like TFS keeps track of checkout times so there probably is no way to know for sure but I’d estimate that it has taken me an average of 10h/week (not including the time spent working on the XNA game I made recently too!) for 10 months which is about:

10 * 4 * 10 = 400 hrs (or 16.7 days)

If I was billing at $90/h that would be $32,400!

Of which I’ve made approximately $0 total. I’m hoping there will be some intangible benefits down the road but the main reasons why I did it were A.) to satisfy my thoughts on how to make a code generator and B.) I wanted to make a website and resented writing the data access layer by hand. So basically it’s taken me almost a year to write some code that would have taken me just a couple days if I wasn’t so damn crazy. Talk about taking the long route to solve a problem!

After squaring up the VB side of things I need to make sure the Getting Started doc is better and I have to update the wiki pages so they're in line with the current version. This is pretty exciting! Be sure to tell your friends!

Labels: , , , , , ,

Friday, July 27, 2007

Ghetto Alarm

Last night I found myself without a cellphone or an alarm clock, I needed to get up by 8am so I had to come up with something. Of course, being the nerd that I am, I decided to write up a quick alarm clock application before I fell asleep. I just plugged in my laptop and changed my power settings so that it would never go into sleep mode and executed the following program:

using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;
using System.Runtime.InteropServices;

namespace GhettoAlarm
class Program
public static extern bool Beep(UInt32 frequency, UInt32 duration);

static void Main(string[] args)
DateTime alarm = new DateTime(2007, 7, 27, 8, 0, 0);
while (true)
Console.Write("\rTime: " + DateTime.Now.ToShortTimeString());
if (DateTime.Now > alarm)
Beep(500, 250);

Needless to say it worked perfect! I'm not sure what would have happened if the laptop power mode was set to fall asleep but I think it probably would still work.

I think the interesting thing here though is how easy it is to just write up a little something to do what you want in C#. I know there is some talk going around about the new dynamic languages and how they are great for rapid prototyping but I just want to say with C# you can change which line of code is executing on the fly, change it and recompile on the fly and even change values in memory on the fly. I'm pretty happy with the rapid prototyping capabilities of C# and I'm a little skeptical of the value of dynamic languages in general... One of these days I'll have to try python again before being too certain about this though.

Monday, July 09, 2007

CodeSnippetExpression is evil

By the way, to anyone thinking about doing some CodeDom development be sure to avoid the CodeSnippetExpression class like the plague. It turns out there is (at least) two legitimate usages of the class that I have encountered but otherwise it is evil.

I can say this because I just spent a couple weeks re-doing all of my templates so that the CodeSnippetExpression class was completely removed from my templates. This was only the beginning of the headache of making my CodeDom code language agnostic but it was the most painful. For the record I'll add my two legitimate usages of snippets and some examples of how I got burned by just saying to myself "Meh, I'll only be generating C# anyway...".

Legitimate use #1 - Calling the default base class constructor

CodeConstructor c = new CodeConstructor();
c.Name = "Example";
c.Attributes = MemberAttributes.Public;
c.BaseConstructorArgs.Add(new CodeSnippetExpression(""));

Example() : base() { }

For some reason you have to add a parameter to the BaseConstructorArgs collection to get this functionality and if you want to call the parameterless constructor you have to add an expression that generates nothing. I suppose you *might* be able to use the CodePrimitiveExpression(null) but I haven't tried that. I can confirm that the empty CodeSnippetExpression("") does work however.

Legitimate use #2 - Fake while loops

CodeIterationStatement i = new CodeIterationStatement(
new CodeSnippetStatement(""),
new CodePrimitiveExpression(true),
new CodeSnippetStatement(""));

for(;true;) { }

Now I realize that this looks stupid but I assure you, it's legitimate. It turns out there is no "while" loop in the CodeDom all you get is the CodeIterationStatement which in turn generates a for loop. However if you think about it what is a for loop? Essentially you're saying "for some initial value, while some condition is true, do this". A for loop with only the condition evaluation is essentially a while loop and the only way to do that with the CodeDom is to print out empty snippets.

So it's ironic that the only legitimate usages of the CodeSnippetExpression objects are simply empty strings. This makes me think that they're utterly evil and should instead be replaced with a CodeEmptyExpression() object or something similar. I welcome any comments about how else they could be properly used.

Here are a few simple examples about how I decided to use them then found it to be very painful to undo them:

Bad idea: "using"
Yes I know the "using" feature in C# is lovely and wonderful and all of that but in reality it is simply syntactic sugar. It is not part of the underlying .net framework.

Proper Alternative:
IDisposable d = ...;
//do something to d.

This is essentially what the "using" statement is doing under the hood. Unfortunately you have to implement this using the CodeDom this robustly. There are no language agnostic shortcuts!

Bad Idea: "anonymous delegates"
Anonymous delegates, oh anonymous delegates, how I love thee... unfortunately you are simply an illusion! Once again more C# syntactic sugar. What's going on under the hood is that a private member is getting generated and a delegate pointing to that method is being generated for you. You're going to have to do it all yourself! Additionally, simply passing method names as parameters for delegates isn't good enough either, more damn syntactic sugar! You're going to have to do it all the hard way.

Proper Alternative:
void Test()
EventHandler e = new EventHandler(Test);
e.Invoke(this, EventArgs.Empty);
void MyDelegateMethod(object sender, EventArgs e)
//do something.

This goes for asynchronous methods as well. In stead of simply passing in Method names for parameters you'll need to use the CodeObjectCreate class to create an instance of your delegate (EventHandler or whatever) and pass in the Method name to that. Make sure you use all the proper CodeDom classes to get this language agnostic.

Those were really the biggest gotchas that I found out. You should start out assuming you're going to be generating for multiple languages and do it right the first time. Be sure to find the right class for the right job too!

Tuesday, June 19, 2007

Nearing v2.0 Release

I only have 2.5 work items left before the release of v2.0. And I may defer one completely. I'm having a bit of a hangup regarding whether or not to implement a feature where you can generate E# based on a database schema. It's not the technical challenge of it, though it would be a fair amount of work, I have done it before and know exactly what it would take to do it again. It's the whole idea of it that I'm debating with. Yesterday at work I was giving a Lunch-and-Learn presentation about code generation and one slide of my power point presentation I have labeled "Source of Truth". This slide prompts me to talk about how I view the meta data as a source of truth and, really, this is the main difference between NBusiness and all other code generators. I'll explain what I mean a little better.

To me with code generation it is possible to end up with a somewhat schizophrenic code generator when it comes to keeping track of it's meta data. This is what I mean by the "Source of Truth", where does the meta-data really reside? For most other code generators they start with extracting the schema from a database as the source of truth and this is the primary problem. However, I question the value of pulling in the schema from the database as a source of truth.

The main problem is really that, since you're pulling it in from a database you are trying to map Objects based on a relational model. This is a well known process known as ORM and as we all know it is a very complicated and tricky problem. However, no matter how well you do ORM, it simply isn't possible to express everything you need in an object in your relational database (at least in a way that translates into an interpretable schema) and inevitably there is a need to customize or add to your meta data. And here is where the problems begin, you have two sources of truth suddenly. How do you handle it so they merge together smoothly, how do you keep customizations while trying to recognize changes in the database, how do you effectively retrieve a subset of the database schema, how do you give things a more friendly name while trying to keep them mapped to the original column and more questions? I mean most of these problems have solutions but they are complicated and difficult and imperfect at best. You might include not allowing customization at all and just forcing the user to deal, you could use some mechanism to merge customizations in with extracted schema meta data.

SO with all that said I should explain how NBusiness differs. NBusiness is a reversal of where the source of truth resides because your E# files are actually your source of truth. They are the 1 and only source of meta data, and to complete the reversal it turns out it is FAR easier to generate your database schema based on objects than the reverse. You could say ROM is much easier than ORM. So for starting up a new project, hands down I'd say the model of authoring your meta-data (entities) without being dependent on a database for extracting schema is far superior. What it comes down to is tooling at that point, which is where I believe NBusiness is unique and valuable. That is, rather than a graphical Visio style development tool you have E#, which is a very easy and intuitive (or strives to be!) language to describe meta data. I think this is why most people have gone the ORM route, because most databases already have a rich and familiar tool for creating tables and columns and relationships, why reinvent the wheel? Well it's my main supposition that E# turns out to be much easier and productive for describing authoring meta-data and you also end up becoming independent from the database as a source of truth which is very valuable.

While I believe all this is true, there is still the situation where you have an existing database. Also, it may be fairly large and manually creating all these entities may be more work than you really want to have to do. That alone could be a critical enough point where using a different tool than NBusiness is a superior choice. So it still might make sense to have the capability to generate some entities based on a schema to really get you started at least. If I did it I'd do it with the focus on keeping your entities as your primary source of truth still. I sort of want to do it but I don't want to degenerate into just another ORM tool. Though, I suppose, if I provide that feature than it will really put my theory to the test. If you have the ability to generate your entities from a database AND the ability to define your entities as the single source of truth then which ever method is truly superior will be the way people use it. If authoring E# manually is not actually better than that tool will be helpful, if it is then it will be helpful sometimes and not other times.

I think I'm sort of talking myself into this even though I set out to talk myself out of it! Dang it.

Maybe I should speak to how I envision NBusiness to handle the situation where you have a set of entities that do not map 1:1 to a database. I imagine this problem will crop up often and I haven't fully tested out the system yet but this is how I imagine it will go. Essentially, when you generate your code you may optionally generate SQL, this sql you get for free and if you're working on an existing project it's really helpful. However if you're not and you have an existing database you may end up in a situation where you don't get to use any of that sql at all. So with NBusiness all of the code it generates (with the default templates and the default framework) you end up with code that uses Stored Procedures as the bridge between the objects and the relational data. All NBusiness code ends up calling a stored procedure such as "PurchaseCollectionFetchByPerson", which when authored would give you a PersonId parameter and expect a set of data related to a collection of Purchase objects. That stored procedure may query tables that look nothing like the purchase entity you have declared so it is the responsibility of the stored procedure to be that bridge. You would have to manually create this in this circumstance.

This get's us 90% of the way there very effectively I think, optimally even. Though there is still the situation where you have a terrible database schema that is not what you want to map to your entities 1:1 AND you don't have the ability to create stored procedures. Maybe your database doesn't have the capability (like older versions of MySql) or perhaps it's just a policy of your company for some reason. For this, unfortunately I'm not sure what to do exactly. I can think of two solutions, one would be to create a custom database provider and translate those stored proc names into some sort of sql but this stinks of a hack. The other solution might be some sort of [Attribute] system in your entity definitions that tell the code generator how to formulate non-standard queries. Actually, I rather like that idea... It probably wouldn't be that hard at all. Well there's something for me to noodle on for a while. v3.0 maybe, LOL.

Anyway, back to enjoying my summer afternoon (or just before noon)!

Labels: , , ,

Thursday, June 07, 2007

The 5 Laws of Code Generation

A colleague of mine from Magenic has written an blog about code generation which summarizes a conversation we've been having:


I really like this post because it nicely lists all of the aspects of code generation that I have been thinking about but haven't been able to put into words. What's nice is that I've tried to address each of these problems with NBusiness in one way or another. Some of these solutions are somewhat unique, especially in terms of the Template system. The following are descriptions of how NBusines resolves the issues outlined by Jon in his post.

CSLA places most of the "heavy lifting" in a set of common base classes

NBusiness does not use CSLA, instead it comes with a light weight, medium-trust, business object framework. It mirrors most of the core features of CSLA as a business object framework but does so in a very different way (due to the need for medium trust). However there is no reason why NBusiness couldn't be used in conjunction with CSLA (or any other framework) since it is template driven but I wanted the official distribution to be completely self contained for now.

I have some fairly specific "technical requirements" that I think that code generation has to to provide:

To me this list is like the "10 commandments" of code generation. Only there are 5, so let's go with the "5 laws of code generation". I'll summarize his list and try to explain how NBusiness conforms to these laws.

1. Code generation is controlled through modifiable templates.

In NBusiness everything is generated by a template. Rather than using any of the existing template schemes (as far as I know) NBusiness takes a unique, more object oriented, approach. The templates themselves are actual classes that inherit from a Template base class. They contain a single method with the signature "ResourceFile[] Generate(Entity[] entities)". The resource files contain a byte[] and path for the resulting files (you could concievably generate images or other binary resources based on entities this way). Of course what goes on in the Generate method is completely up to the template developer. Furthermore, this scheme allows for inheritance in templates. So, for example, if you decided to use one of the templates that came with NBusiness but you just don't quite like one line of the output, you can create your own template and override an appropriate virtual method and customize the output yourself without destroying the original template. This more object oriented approach brings forth some interesting possiblities.

There is also a notion of template "dependencies", which allows you to selectively generate code if an Entity contains a template that fulfills a dependency. For example if you have a template for generating an EntityBaseCollection object, you'll naturally need an EntityBase object to be generated as well. Therefore the EntityBaseCollectionTemplate is dependent on the EntityBaseTemplate. This problem arises quite a bit and this solution is very helpful for solving these issues. Without it you end up getting to a point where you might as well just make one giant template and apply it to all of your entities rather than selectively implementing only what a particular entity needs.

2. Code generation is done during the build process.

This law is one of NBusiness' best features. Not only is done during the build process the NBusiness engine is actually a compiler. In fact the artifact that NBusiness produces is a class library rather than simply a set of classes to be loaded into a seperate class library project. In v2.0 this is automated fully by means of Visual Studio Integration. You litterally have an E# project that can be referenced by your other projects as intuitively as you would any normal class library.

3. Code generation is done from an intermediate format.

This law is very important to me. NBusiness goes one step further than what Jon is suggesting as the optimal solution however. Jon suggests using XML that is (optionally) generated from a database schema, which I couldn't agree more as the right solution to the problem. However instead of XML NBusiness features an Entity Definition Language known as E#. It is essentially the same thing: a set of meta data that describes your entities. However it is my belief that XML really isn't as developer friendly as what is really needed. Rather than creating all sorts of grahpical development tools that allows a developer to manipulate this XML what is really optimal is to have a model that is the same as what most developers are currently most familiar and productive with, which is code. E# (in v2.0) has visual studio integration which comes complete with syntax hilighting and intellisense and allows a developer to build their entity meta data in a style similar to what they are already familiar with. The only barrier is that you have to learn "yet another language" to do it. E# strives to be extremely simple by having syntax similar to C# and it is also fairly VB-ish in how the keywords flow. Intellisense helps quite a bit as well of course.

4. The intermediate format is under source code control with versioning.

This law is helpful for restricting the definition of what the intermediate format is. I would suggest an ammendment to this law that states the intermediate format must also be such that it is compatible with "merging" features of source code control. This will promote the ideal of having this data as plain text rather than a custom binary format. Of course, since E# is defined in source code files it meets this law fully.

5. The code generation templates are also under source code control.

Again I would ammend this with the mergability rules from #4. In the case of NBusiness the base templates are source code files currently under source control at CodePlex while custom templates would simply be class files somewhere in your project, which is absolutely compatible with source code control.

The next section of Jon's post addresses more of the features required of a business object framework as they relate to code generators. This is an important distinction because, while NBusiness has such a framework, it could certainly be used to target any 3rd party business object framework which may or may not conform to these rules. This is mostly a concern of the templates but there are definitely accommodations for a particular style that need to be taken in the framework itself. NBusiness' default business object framework takes the second style, where partial classes are used to accomodate custom code.

To me the main benefit of this is the cleanliness of the resulting object model. While I'd concede that the base/child style is functionally fine, I would argue that having fewer classes is far superior. The tradeoff, is that you end up having to expose many events in order to provide extensibility points for the other parts but this is ok to me. In fact in NBusiness there is a feature called "actions" that allow you to define the behavior of an entity as a method. These actions can optionally be attached to these extensibility events and even be automatically executed asynchronously. This action method is implemented by the developer in a partial class or even as a static method on another class entirely. So these events would be there anyway.

An example of this might be a "TransferMoney" action on a BankAccount entity. Or a Log method attached to the "persisting" event, etc. You can envision how useful it is to define your entites behaviours in the entity definition itself. The architect can define everything and the code monkey can implement it.

So with Jon's post in mind I propose the 5 laws of code generation:

  1. Code generation is controlled through modifiable templates.
  2. Code generation is done during the build process.
  3. Code generation is done from an intermediate format.
  4. The intermediate format must be versionable and mergeable.
  5. The code generation templates must be versionable and mergeable.