Tuesday, June 19, 2007

Nearing v2.0 Release

I only have 2.5 work items left before the release of v2.0. And I may defer one completely. I'm having a bit of a hangup regarding whether or not to implement a feature where you can generate E# based on a database schema. It's not the technical challenge of it, though it would be a fair amount of work, I have done it before and know exactly what it would take to do it again. It's the whole idea of it that I'm debating with. Yesterday at work I was giving a Lunch-and-Learn presentation about code generation and one slide of my power point presentation I have labeled "Source of Truth". This slide prompts me to talk about how I view the meta data as a source of truth and, really, this is the main difference between NBusiness and all other code generators. I'll explain what I mean a little better.

To me with code generation it is possible to end up with a somewhat schizophrenic code generator when it comes to keeping track of it's meta data. This is what I mean by the "Source of Truth", where does the meta-data really reside? For most other code generators they start with extracting the schema from a database as the source of truth and this is the primary problem. However, I question the value of pulling in the schema from the database as a source of truth.

The main problem is really that, since you're pulling it in from a database you are trying to map Objects based on a relational model. This is a well known process known as ORM and as we all know it is a very complicated and tricky problem. However, no matter how well you do ORM, it simply isn't possible to express everything you need in an object in your relational database (at least in a way that translates into an interpretable schema) and inevitably there is a need to customize or add to your meta data. And here is where the problems begin, you have two sources of truth suddenly. How do you handle it so they merge together smoothly, how do you keep customizations while trying to recognize changes in the database, how do you effectively retrieve a subset of the database schema, how do you give things a more friendly name while trying to keep them mapped to the original column and more questions? I mean most of these problems have solutions but they are complicated and difficult and imperfect at best. You might include not allowing customization at all and just forcing the user to deal, you could use some mechanism to merge customizations in with extracted schema meta data.

SO with all that said I should explain how NBusiness differs. NBusiness is a reversal of where the source of truth resides because your E# files are actually your source of truth. They are the 1 and only source of meta data, and to complete the reversal it turns out it is FAR easier to generate your database schema based on objects than the reverse. You could say ROM is much easier than ORM. So for starting up a new project, hands down I'd say the model of authoring your meta-data (entities) without being dependent on a database for extracting schema is far superior. What it comes down to is tooling at that point, which is where I believe NBusiness is unique and valuable. That is, rather than a graphical Visio style development tool you have E#, which is a very easy and intuitive (or strives to be!) language to describe meta data. I think this is why most people have gone the ORM route, because most databases already have a rich and familiar tool for creating tables and columns and relationships, why reinvent the wheel? Well it's my main supposition that E# turns out to be much easier and productive for describing authoring meta-data and you also end up becoming independent from the database as a source of truth which is very valuable.

While I believe all this is true, there is still the situation where you have an existing database. Also, it may be fairly large and manually creating all these entities may be more work than you really want to have to do. That alone could be a critical enough point where using a different tool than NBusiness is a superior choice. So it still might make sense to have the capability to generate some entities based on a schema to really get you started at least. If I did it I'd do it with the focus on keeping your entities as your primary source of truth still. I sort of want to do it but I don't want to degenerate into just another ORM tool. Though, I suppose, if I provide that feature than it will really put my theory to the test. If you have the ability to generate your entities from a database AND the ability to define your entities as the single source of truth then which ever method is truly superior will be the way people use it. If authoring E# manually is not actually better than that tool will be helpful, if it is then it will be helpful sometimes and not other times.

I think I'm sort of talking myself into this even though I set out to talk myself out of it! Dang it.

Maybe I should speak to how I envision NBusiness to handle the situation where you have a set of entities that do not map 1:1 to a database. I imagine this problem will crop up often and I haven't fully tested out the system yet but this is how I imagine it will go. Essentially, when you generate your code you may optionally generate SQL, this sql you get for free and if you're working on an existing project it's really helpful. However if you're not and you have an existing database you may end up in a situation where you don't get to use any of that sql at all. So with NBusiness all of the code it generates (with the default templates and the default framework) you end up with code that uses Stored Procedures as the bridge between the objects and the relational data. All NBusiness code ends up calling a stored procedure such as "PurchaseCollectionFetchByPerson", which when authored would give you a PersonId parameter and expect a set of data related to a collection of Purchase objects. That stored procedure may query tables that look nothing like the purchase entity you have declared so it is the responsibility of the stored procedure to be that bridge. You would have to manually create this in this circumstance.

This get's us 90% of the way there very effectively I think, optimally even. Though there is still the situation where you have a terrible database schema that is not what you want to map to your entities 1:1 AND you don't have the ability to create stored procedures. Maybe your database doesn't have the capability (like older versions of MySql) or perhaps it's just a policy of your company for some reason. For this, unfortunately I'm not sure what to do exactly. I can think of two solutions, one would be to create a custom database provider and translate those stored proc names into some sort of sql but this stinks of a hack. The other solution might be some sort of [Attribute] system in your entity definitions that tell the code generator how to formulate non-standard queries. Actually, I rather like that idea... It probably wouldn't be that hard at all. Well there's something for me to noodle on for a while. v3.0 maybe, LOL.

Anyway, back to enjoying my summer afternoon (or just before noon)!

Labels: , , ,


Post a Comment

Links to this post:

Create a Link

<< Home