Tag Archives: Patterns

Patterns

Presentation Applying “web scale” patterns in the bol.com back office

Codemotion web scale pattern presentationThis week at Codemotion I gave a presentation on web scale patterns and how we apply them in the bol.com back office services. Codemotion is the biggest tech conference in Italy and one of the most important in Europe.

The presentation shows how we go from business goals to software patterns. The following patterns were covered: CQRS, event sourcing, polyglot persistence and micro services.


Presenting at Codemotion next week on web scale patterns

Peter Paul van de Beek CodemotionNext week I’ll be presenting in Amsterdam at Codemotion on web scale patterns in bol.com back office services. Codemotion is the biggest tech conference in Italy and one of the most important in Europe, with a network of more than 30k developers.

The title of my presentation is: “Applying “web scale” patterns in the bol.com back office”. In the session, we show you how we use “web scale” patterns to achieve scalability and flexibility in our back office software. We will guide you through how we apply patterns like CQRS, event sourcing, polyglot persistence and micro services to solve puzzles in our back office services. Interesting is that in our experience we don’t need these just to solve a technical problem. It helps us to solve some of our business problems!

See you on May 16 and 17 in Amsterdam. There is a short interview with me on Codemotion world.

web scale patterns in the bol.com back office – Mixed SQL – NoSQL

In the previous weeks, we started a series of blog posts that show you how we use “web scale” patterns to achieve scalability and flexibility in our back office software. The previous patterns discussed were Event Sourcing and CQRS. This week we will dive into mixed SQL – NoSQL. Showing you how this doesn’t just solve a technical problem, they help us solve our business problems!

Mixed SQL – NoSQL

Where needed in our services we are moving away from pure SQL. We create a mix with other types of storage. So we are using NoSQL (Not only SQL). One could also call this polyglot persistence. The notion that your application can write to or query multiple databases or one database with multiple models. It uses the same idea as polyglot programming. This expresses the idea that applications should be written in a mix of languages to take advantage of the fact that different languages are suitable for tackling different problems.

RTN – Billing platform for our retailers

RTN stores all kinds of transactions to charge and pay partners in our LvB (Fulfilment by bol.com) operations. In this part of our operation, we store and fulfil products in our warehouses for retailers that sell their goods on our platform.

The transactions to create invoices for our LvB partners stems from a number of services. We have all kinds of different attributes we want to account for to know why decisions have been made and for auditing purposes. These attributes depend on the transaction type. It was decided that the attributes wouldn’t be a part of the transactions table since they are only filled for a part of the records.

To accommodate for the attributes that depend on the transaction type we created an additional column in the table that is able to store key-value pairs in JSON format. The use of a pure SQL solution would have resulted in a weak design. As would the use of a pure NoSQL. In these cases, they work great together.

FNK – Warehouse orders

FNK processes our customer orders to create warehouse orders. It determines the warehouse that will fulfil the customer demand and instructs the warehouse to fulfil the order. Besides regular warehouses, it also communicates with our warehouse for digital products (e-books and software downloads) and with retailers that sell their products on our platform and take care of the fulfilment themselves.

These retailers have requirements that differ from the other warehouses. To accommodate these while avoiding a to a specific part in this services we introduced an additional column that stores XML. This mixture of SQL (one table for all warehouse orders) and NoSQL (stored XML) results in a simple model that can handle requirements that are only needed for a part of the orders. Since the data in the XML is hardly needed in this service but mostly in downstream services, there are no drawbacks on performance.

What we learned

The NoSQL parts in these mixed data stores are mostly used to read from. If you need to specifically filter on these of have a requirement to use them in joins performance will degrade.

Next in web scale patterns in the bol.com back office

In the next week’s episode on the following subject will be published:

  • Micro services
web scale patterns in the bol.com back office

web scale patterns in the bol.com back office – Event Sourcing

Last week we started a series of blog posts we show you how we use “web scale” patterns to achieve scalability and flexibility in our back office software. Last week’s pattern we discussed was CQRS. This week we will dive into Event Sourcing. Showing you how this doesn’t just solve a technical problem, they help us solve our business problems!

Event Sourcing

The idea of Event Sourcing is that every change to the state of a system is captured in sequence and that these events can be used to determine the current state. Consequently, the state of the system for any point in time can be determined by replaying the events. The structure of the service changes from storing state to storing events.

The most obvious that we gain by using Event Sourcing is that we have a log of all the changes. We can see everything that happened. This enables us to:

  • Do a complete rebuild;
  • Determine the state of the system at any point in time;
  • Event replay – Compute the consequences of a change in a past event of recalculate the consecutive states based on the proper sequence of events (in case messages in an asynchronous communication weren’t received in the proper order).

Using Event Sourcing can feel a little bit awkward for some developers. However, it offers a variety of opportunities. One could replay the events on a test environment to see exactly what happened on pro, while you have the ability to stop, rewind and replay the events running a debugger. This provides also a way to do parallel testing before promoting an upgrade to production.

Where do we use it at bol.com?

web scale patterns in the bol.com back officeOne of the examples where we use Event Sourcing is Condition Management and especially the calculations of accruals and invoices for (purchasing) conditions. A large set of our purchasing conditions is based on either purchasing amounts or values and sales amounts and values. In general these purchasing conditions have to attributed to (sets of) single products, product categories, suppliers and brands.

Storing the events that represent the purchase and sales of goods allows us to implement functionality that would be very hard to develop if we wouldn’t. Typically a purchasing condition isn’t agreed with a supplier of a brand on the first of January. While it could be valid from the first of January. The Event Sourcing model allows us to handle conditions that are entered into the system somewhere in March or April that are valid from the first of January. These conditions will be handled by passing all the events from the start date and the appropriate accruals and invoice can be created.

With the Event Sourcing model, we are also more loosely coupled to the source services for purchasing and sales. Our calculations can handle events that are captured out of sequence or even very late. Condition values are still calculated properly and handled as accounting and controlling have prescribed.
For the future, we are planning to implement scenario run through and comparisons. This would support our buyers while negotiating with suppliers.

Next in web scale patterns in the bol.com back office

In the next week’s episodes on the following subjects will be published:

web scale patterns in the bol.com back office – CQRS

web scale patterns in the bol.com back office – CQRS

In this series of blog posts, we show you how we use “web scale” patterns to achieve scalability and flexibility in our back office software. We will guide you through how we apply patterns like CQRS, event sourcing and micro services to solve puzzles in our back office services. These patterns don’t just solve a technical problem, they help us solve our business problems!

We need web scale in the back office since more and more functionality from the back office is needed on the web site to offer better service to our customers. For example, more parts of our web shop do request on our stock levels and warehouse configuration to determine how fast product can be delivered to our customers and with what options. Consequently, the services that know our stocks levels and warehouse configuration also have to be scaled to handle these volumes. To enable this we don’t just need more hardware, we also need to apply patterns to our services to create a proper structure.

CQRS

CQRS is short for Command Query Responsibility Segregation. At the core of CQRS is the notion that a different model can be used to alter data than the model that is used to query data. Updating and reading information have different requirements on a model. There are enough cases where it serves to split these. The downside of this separation is that it introduces complexity. So this pattern should be applied with caution.

The most common approach for people to interact with data in a service or system is CRUD. Create, Read, Update and Delete are the four basic operations on persistent storage. The term was likely popularised by James Martin in his 1983 book Managing the Database environment. Although there exist other variations like BREAD and MADS, CRUD is widely used in systems development.

When a need arises for multiple representations of information and users interact with these multiple representations, we need something that extends CRUD. This because the model to access the data tends to be split over several layers and becomes overly complicated.

What CQRS adds

CQRS introduces a split into separate models for update and display, Command and Query respectively. The rationale for this is that for many problems in more complex domains having the same model for commands and queries leads to a more complex model. A model that does neither well.

Where do we use it at bol.com?

One of the examples of where we use CQRS in the back office services at bol.com is in our Inventory Management. Inventory Management handles all updates on stock levels and serves them to several services in out landscape including our web shop.

The updates of stock levels come from our warehouse management and include reservations based on customer orders, shipments and received goods. The queries on the stock level originate in the web shop, check out and fulfilment network. As you can imagine these queries have quite a different profile compared to the updates. Besides that, the number of queries far outreaches the number of updates.

Given these different requirements we decided to split command (updates) and query for inventory management. All updates are handled by a technically isolated part of the service. Stock levels are served by other services by another isolated part.

Implementation

web scale patterns in the bol.com back office – CQRSThe part that handles the updates has several models. The incoming changes like the shipments and received goods have to be handled in for example stock mutations, stock levels and stock valuation. These models receive updates and process them to a new stock level and stock valuation. Once a new stock level is calculated, it is published on a messaging queue to the query part. This message is also consumed by other services that need these.

The query part is a simple single table. The messages from the update part are stored in this table and there is no additional logic or processing. Queries from other services are handled by a REST interface. Due to this design, this call has a very high cache hit ratio. Which of course leverages performance.

Next in web scale patterns in the bol.com back office

In the next week’s episodes on the following subjects will be published:

Presenting on pragmatic microservices GOTO Night Thursday, May 12, 2016

goto nights pragmatic microservicesOn Thursday, May 12, 2016 I will be presenting on pragmatic microservices at the GOTO Night organised at bol.com. The presentation will be the support act for Randy Shoup. Check some of his previous presentations on SlideShare.

Pragmatic microservices

We have been around in e-commerce for years. However compared to other companies we’re young. Some would say we are in the scale up phase. In a number of ways we are experiencing a rapid growth. What does our IT need to stay innovative and scale to enable all this? What are the tradeoffs that are made for innovation in IT?

This year we won the Best Web Shop award because of our “efforts to get the difficult to achieve basics right that make the difference for customers”. IT has a large role in achieving this, at the scale of a web shop like bol.com. Did (micro)service make the difference to achieve this?

At bol.com we have a pragmatic, business value driven approach to (micro)services. In this presentation we share insights and the tradeoffs we made so IT enables to scale and innovate.

Presentation Pragmatic (Micro)Services

Here is the presentation I used:

Experiences with Holacracy

There is a growing number of books on holacracy. One of the first on this subject without even coining the term was Eckart’s Notes. Also the ones on Semco (like Semco style and Maverick!) are gaining popularity. These all describe case studies, where Reinventing Organisations shows development stages in organization. These in turn are based on literature and case studies like the one that is described in Eckart’s Notes.

Since we are experimenting with holacracy at Bol.com I recently read Getting Teams Done. It gives a great introductions and offers a practical approach.

Holacracy cases

Recently there were some case studies and testimonials of the use of holacracy published at Medium.
Path to holacracy

The Dutch design company Concept7 has experimented with a number of ‘management styles’ over the years. Their objective has always been clear: to foster freedom and empowerment in their employees. The entire staff has been working with holacracy for a few years now. Lauren shares her story.

The founder of Voys, Mark Vetter, has always believed in self-organisation. He claims holacracy has brought us accountability, entrepreneurship, and faster evolution. Check Mark’s story on getting things done in a company. The Voys way is published in a book on the way they are organised.

The founders of the Dutch ‘user experience’ company Valsplat believe it is utmost important to bring your whole self to work, or even express or develop your talents. Nils claims holacracy has provided me with tools that help me focus. He states:

Holacracy helps us to become the best possible version of ourselves

Boek – Getting Teams Done – Spark

Getting Teams Done - SparkHet boek Getting Teams Done is een leuk introductie in Holacracy. Dit boek introduceert de methode waarmee leidinggevenden, teamleiders managers en zelfsturende professionals de teamproductiviteit naar een hoger niveau kunnen tillen. Holacracy wordt bijvoorbeeld ook gebruikt als organisatievorm bij Zappos (lees ook Delivering Happiness).

Getting Teams Done is geschreven in de stijl van Het Doel / The Goal en The Phoenix Project. De romanvorm wordt daarbij in dit geval afgewisseld met een non-fictie deel waarin wordt uitgelegd hoe Holacracy werkt en waar het een oplossing biedt. Daarnaast wordt in dat deel steeds het proces en de ondersteunende technieken uitgelegd.

Het boek Getting Teams Done is een aanrader voor iedereen die geinteresseerd is in Holacracy en zeker voor diegene die werkt voor een organisatie waar Holacracy of een variant als Spark ingevoerd wordt. Het invoeren van Holacracy of Spark vraag een zeer sterke discipline. Daarbij is het handig om hulp van buiten te hebben die buiten de inhoudelijke en andere discussies staat. Op die manier kan er sneller mee aan de slag gegaan worden en kunnen de resultaten eerder worden bereikt.

Velocity 2015 Amsterdam

Thursday was a very interesting day for me at Velocity 2015 Amsterdam, build resilient systems at scale. It is one of the best conferences I attended in the last years. Using some quote’s and bullets I’ll give a little insight.

On retro’s, post mortems, etc

Lindsay Holmwood showed that what goes wrong in retrospectives, post mortems and the like is mostly based on:

  • Confirmation bias – aka ignore alternatives
  • Hindsight bias – aka – alter memories to fit a narrative. Talk about events with the knowledge of the outcome.

To overcome these we could use techniques like: Take opposing viewpoints (on purpose, to investigate things), contrarian thinking, let people explain stuff in terms of foresight and all kinds of sharing information.
In short for this to work we need a safe environment where people can speak up. Starting from the believe that everyone did the best they could. And always keep in mind that there is a difference between work as imagined and works as done.

Optimizing teams in a distributed world – Conway’s 3 other laws

Conway’s Law stems fro the greater part from his 1967 paper: How do committees invent.
The slides and all references mentioned in the presentation.

  1. Whose structure is a copy of the organisation’s structures. To put it different: Communication dictates design. Also check The Mythical Man Month and Dunbar’s number. So manage communication between teams.
  2. – Doing it over – There is never enough time to do something right, but there is always enough time to do it over. Engineering and architecture are always about: Trade offs. Also check: Satisfying vs Sacrificing. So remember it is a continuous process.
  3. If you or your team cannot explain all the code in your release package your release is too large.

  4. Homomorphism – If you have 4 groups working on a compiler, you’ll get a 4-pass compiler. So organise teams in order to achieve what you want (around business capabilities).
  5. Disintegration – The bigger they are, the harder they fall. Time is against large projects and teams. Aim for a scope that supports a release cycle of two weeks or less. So keep your teams as small as necessary.

It is better to be too small than too big.

We are all DevOps

One of the best talks on DevOps in the Etsy world by Katherine Daniels.

On hiring:

It is easier to teach someone a new technology skill, than to teach someone not to be an asshole.

Continuous Delivery at bol.com

Last month two of our software engineers Mihaela Tunaru and Mary Gouseti were invited to give a presentation of how continuous delivery is done at bol.com. The presentation gives a good insight in the state of continuous delivery at bol.com from a software engineering perspective.

In case you want to know more from the operations perspective check Mayfly on GitHub and the presentation below. Maarten Dirkse gave a talk Docker your user stories using Mayfly.

Mayfly is a development platform built by bol.com. Mayfly speeds up your service development by wrapping your scrum user story code in containers, testing it in an isolated, production-like environment and automatically enforcing your Definition of Done.