CrudQRS, a pragmatic approach to CRUD and CQRS using LightSwitch

After last month’s tales of advanced client-side customization, I originally intended to share a server-side tale about “Nancy”‘s LightSwitch application.  However, since HTML5 has entered the LightSwitch town as the new deputy, the MSDN team and I rapidly changed course; long story short: it’ll be a while before I will cover this in-depth on MSDN so I decided to throw a short summary on my blog, share the idea, perhaps start some discussions, …  By the way, I love experimenting with LightSwitch to see how far I can push it.  To quote Michael Washington from a mail after I wrote about MyBizz Portal (which happens to be the v1.0 of the application that’s running at Nancy’s company): “experiments that take LightSwitch to the next level… Way beyond…”

Short recap: what’s REST?

RESTful services, OData, CRUD, entity-based UI, I’m not sure what this is called when talking about architectural patterns.  The idea is that your application consist out of multiple “resources” or “entities”.   It’s custom to think of these as the “nouns” in the story of your business: “Customer” “Invoice” “Shop” etc.  Traditionally, there are 4 verbs in the story: “Create”, “Read”, “Update” and “Delete”.  Both the nouns and the verbs are found throughout the application:

  • Client side: “CustomerList”, “Search invoices”, “Add a new Contact”, etc
  • In the transport protocol (OData): get, post, put, delete (verbs) and URIs (nouns)
  • In the database: “Update dbo.Customers set (..) where (..)”

This traditional approach offers quite some benefits, but has some downsides too:

  • About the database: you’ll traditionally find a normalized database that’s  optimized for all 4 verbs equally: creating, updating, reading and (not always) deleting data.  A normalized database is great for supporting applications where all 4 verbs are equally important, but this is not always the case.  Think about scalability: some systems (Twitter & Facebook, for example) need a magnitude larger capability for querying, then creating and updating the data.  Think about performance: how long will a query take “that returns the average increase in revenue per site, per month, over the last 15 years, compared to the average money spent on training per employee during the same period”?  When this “reading power” becomes a real requirement, you’ll find benefits in adding a denormalized database to your application.  Welcome to the world of BI (datawarehouse, SSAS, Cubes, ROLAP or MOLAP, etc).  Once you have all this BI power, isn’t it a huge waste not to reuse it in the queries that fill the screens of your client application?
  • The same scalability can be extended to your services too.  If a service can handle all (CRUD) verbs regarding a particular noun, then they can’t scale to reflect the need for “reading power” being magnitudes smaller or larger than the need for the other verbs.
  • Also, being limited to just 4 verbs means it’s hard to capture the intent of the end-user, the business process that drove a certain change on an entity.  A customer isn’t “created”, he/she “joins a mailing list”.  A customer doesn’t “update”, he/she “moves house”.  In the latter case, the business might want to send an info-brochure to the new address to inform the customer of the nearest shop?  Perhaps you can this process is free but only (maximum) once a year, otherwise the customer has to pay some administration costs.  In these cases, there’s a distinct difference between how the application should react when “the customer’s address contained a typo”, “the customer moved” or “the name of the street was changed”.  This difference can’t be expressed by the noun+verb combination “Update Customer”.  By limiting the verbs to a vocabulary of only 4, you’re hiding what really happened.

Short recap: what’s CQRS?

CQRS addresses these scalability issues and issues to capture the intent of the end-user in particular.  CQRS is an architectural pattern that advocates the benefits of having  “Commands” (C) and “Queries” (Q), and clearly splitting the responsibilities of the two (Responsibility Segregation, RS).

You can read more about this architectural pattern on Udi Dahan or Martin Fowler or Greg Young ‘s blogs. Although not *strictly* required, you’ll often see this responsibility segregation reflected front (a “task-based UI”) and end (at least two databases, one normalized and one denormalized) which I would recommend to fully enjoy this powerful architecture.

Notice that this is a totally different approach from CRUD.  There’s no “CUD in CRUD” == “Commands” and “R in CRUD” = “queries” equation to be found.

  • The commands represent actual business processes.  “MoveCustomerCommand” vs the CRUD “UpdateCustomer”.
  • The queries have dedicated services, and sometimes (I recommend it) dedicated databases.

This system has a lot of benefits and doesn’t suffer from the same scalability issues or troubles trying to capture the intend of the user, but it comes with its own set of downsides:

  • It’s more work to set up & maintain.
  • The two databases can be out of sync, resulting in differences when reading the data straight after submitting a command (although end-users have gotten used to their data being async really, think about sending a tweet for example).
  • Introducing a “reinventing-the-wheel” vocabulary (read about SOAP vs REST: Vocabulary re-use vs. its arbitrary extension: HTTP and SOAP)
  • I doubt many end-users will enjoy a UI that has a button for invoking the “MoveCustomerCommand”, followed by a button to invoke the “ChangeCustomerStreetNameCommand” and a button “ChangeCustomerAddressBecauseItContainsTypoCommand”.
  • LightSwitch doesn’t support CQRS out-of-the-box, so you’ll find it very counter-productive to force a LS UI&service to this model.

Introducting CrudQRS, a pragmatic approach to CRUD and CQRS

LightSwitch v1.0 made it rather difficult to blend different LightSwitch apps together, but because of the switch to OData, it’s quite easy to blend the middle-tier of different LightSwitch applications and call one from the other.  To benefit from the advantages CRUD and take advantage of some CQRS benefits, you need:

  1. One very normal LightSwitch app.  This is the CRUD-part.  The “nouns” are domain models that encapsulate the business rules, validation, etc.  However (using some manual plumbing because there’s no “generic” access to the write code extension points), when an entity is saved, it persists this change to the database, but also logs it to a generic auditing table:
  2. One LightSwitch data-service that hosts the “generic auditing table”, it’s largely based on Paul Van Bladel’s articles on generic auditing trails.
  3. One long-running process (of any kind) that takes the entries from the auditing table, and processes them by updating the data in:
  4. One denormalized LightSwitch data-service.  Basically, this service is optimized to provide all the “reading power” you need, in custom reports, in Excell power-pivot tables, in specialized screens (in the normal LS app described in the first item that might have performance issues because there’s a lot of different entities being shown on each line), …

The second data-service might seem like an overhead at first: why wouldn’t you update the denormalized data-source directly?  Well, because

  1. The denormalized data-source is geared towards reading only.  Thus, updating might take a lot of time.  You do not want your end-users to wait for this, hence the need of some kind of asynchroneous process, some kind of queuing system.  I could have chosen many solutions for this (MSMQ for example), but abusing a LightSwitch OData service for this got me up and running in under 15 mins.
  2. You get the added value of having an auditing table for free.  This table can also be useful in the LightSwitch app, to do… well… auditing :-)

And that, my friends, completes the circle of CrudQRS, an architecture made (almost) entirely in LightSwitch, with the advantages of building out CRUD-based applications really fast, but is scalable and optimized for complex queries like a CQRS system would be.

I’m currently experimenting with a second table in my LightSwitch data-service (2.) that hosts a command table, because not everything is easily expressed in CRUD.  These commands can be “instantiated” by the user by creating reusable modal screens.  When I’ve played with this a bit more, I think Nancy’s app will be the pragmatic mixture between CRUD and CQRS that I intended it to be.

Just like “pure-CRUD” or “pure-CQRS”, this semi-in-the-middle-architecture comes with benefits and downsides.  Choosing the “right” architecture or tooling is always a “scenario-dependent” choice.  It worked for Nancy.  I’m sure I’ll have many more applications where it’ll work.  I’m just curious: do you see any applications where this might be the choice of architecture for you?  And: would you be interested if I covered this more in-depth?

Keep rocking LS!

About these ads

7 thoughts on “CrudQRS, a pragmatic approach to CRUD and CQRS using LightSwitch

  1. Thanks, Jan, for reminding me of my roots in Business Process Analysis. I TOTALLY agree that the ridiculous CRUD paradigm is beyond limiting and conveys nothing of what the solution is actually doing. To utterly rely upon the “business logic layer” to use over-wide CRUD Stored Procedures is taking a sledgehammer to crack a walnut. Define your ACTUAL business processes, provide focused, self-explanatory, Stored Procedures to implement them and enable the business layer to call them. For example, to “Launch a Site” requires the ID and Launch Date, not all 22 fields and the UPDATE Site SP to decide, “what just changed”… Duh! Whoever thought CRUD-only was a great idea must be behind other dumb notions such as “the database is just a filing cabinet” (as heard from too many UI developers!). Commands get my FULL support. I’d hope that any reasonable architect would deliberately smash the CRUD-mold and get back to real business system analysis and architecture. I’ll certainly be using Commands, hooked up to LightSwitch with only minimal use of Create and near zero use of UPDATE from the CRUD-set. Though C.A.S.E. technology fell into unwarranted disrepute, the Process Analysis and Process Dependency Analysis facets of Information Engineering that respect a fully ACID Data Model are fully applicable here – note they are implementation-agnostic.
    I look forward to more on your implementation of this very practical (overdue) paradigm.
    Cheers!

  2. Pingback: VB Feeds

  3. Pingback: Windows Azure and Cloud Computing Posts for 8/2/2012+ - Windows Azure Blog

  4. Hey Jan,

    Thanks for putting up this article!

    A couple of thoughts:

    REST and CQRS are not mutually exclusive. The article seems to suggest that this is the case. Apologies if I misinterpret this. Just in case:

    REST describes using a well defined vocabulary to work with resources. Using the TRACE and OPTIONS HTTP methods for example would not be a violation of REST principles. Its about using what the application layer protocol already supports instead of introducing a “reinventing-the-wheel” vocabulary like when using SOAP RPC. (so this is *not* a downside that can be tributed to CQRS)

    CQRS, at its heart, is about using different object models for reading and altering state. Extending from this notion is the introduction of separate service contracts, separate data schemes etc.

    A GET request implementation could certainly benefit of using a different model then say a POST request implementation.

    Capturing the intent of the change can still be achieved using a RESTful api. The ‘resources’ exposed via the REST API don’t necessarily map with the nouns in your ‘command model’ one on one. (POST ‘Move’ could change the address record of a customer).

    Kind regards!

  5. Not a fan – eliminating the CRUD is more important to me. Commands are the first class citizen which lead to a task driven UI. LightSwitch is nice but dataset CRUD centric. Creating a second tier denormalised View Model for the UI to then fires dataset deltas as some form of command seems to be too much of a compromise. Where would the command handles live?

  6. Pingback: CrudQRS, a pragmatic approach to CRUD and CQRS using LightSwitch | Jan Van der Haegen’s blog | Ra Puke Moana

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s