In this series of blog posts, we show you how we use “web scale” patterns to achieve scalability and flexibility in our back office software. We will guide you through how we apply patterns like CQRS, event sourcing and micro services to solve puzzles in our back office services. These patterns don’t just solve a technical problem, they help us solve our business problems!
We need web scale in the back office since more and more functionality from the back office is needed on the web site to offer better service to our customers. For example, more parts of our web shop do request on our stock levels and warehouse configuration to determine how fast product can be delivered to our customers and with what options. Consequently, the services that know our stocks levels and warehouse configuration also have to be scaled to handle these volumes. To enable this we don’t just need more hardware, we also need to apply patterns to our services to create a proper structure.
CQRS is short for Command Query Responsibility Segregation. At the core of CQRS is the notion that a different model can be used to alter data than the model that is used to query data. Updating and reading information have different requirements on a model. There are enough cases where it serves to split these. The downside of this separation is that it introduces complexity. So this pattern should be applied with caution.
The most common approach for people to interact with data in a service or system is CRUD. Create, Read, Update and Delete are the four basic operations on persistent storage. The term was likely popularised by James Martin in his 1983 book Managing the Database environment. Although there exist other variations like BREAD and MADS, CRUD is widely used in systems development.
When a need arises for multiple representations of information and users interact with these multiple representations, we need something that extends CRUD. This because the model to access the data tends to be split over several layers and becomes overly complicated.
What CQRS adds
CQRS introduces a split into separate models for update and display, Command and Query respectively. The rationale for this is that for many problems in more complex domains having the same model for commands and queries leads to a more complex model. A model that does neither well.
Where do we use it at bol.com?
One of the examples of where we use CQRS in the back office services at bol.com is in our Inventory Management. Inventory Management handles all updates on stock levels and serves them to several services in out landscape including our web shop.
The updates of stock levels come from our warehouse management and include reservations based on customer orders, shipments and received goods. The queries on the stock level originate in the web shop, check out and fulfilment network. As you can imagine these queries have quite a different profile compared to the updates. Besides that, the number of queries far outreaches the number of updates.
Given these different requirements we decided to split command (updates) and query for inventory management. All updates are handled by a technically isolated part of the service. Stock levels are served by other services by another isolated part.
The part that handles the updates has several models. The incoming changes like the shipments and received goods have to be handled in for example stock mutations, stock levels and stock valuation. These models receive updates and process them to a new stock level and stock valuation. Once a new stock level is calculated, it is published on a messaging queue to the query part. This message is also consumed by other services that need these.
The query part is a simple single table. The messages from the update part are stored in this table and there is no additional logic or processing. Queries from other services are handled by a REST interface. Due to this design, this call has a very high cache hit ratio. Which of course leverages performance.
Next in web scale patterns in the bol.com back office
In the next week’s episodes on the following subjects will be published: