First episode of the bol.com TechLab podcast

Today we released the first episode of the bol.com TechLab podcast. The subject of this first episode is our Kotlin adoption journey.

In this podcast, we share our experience with you to learn and entertain. Peeking behind the screens of IT and Tech in general at bol.com. Showing you our approach to IT, e-commerce and retail platforms. We have a lot of fun creating the podcast episodes. It is awesome to talk about all the interesting software we create and the innovation we can achieve with our development community.

Our next episode will feature our experience with forecasting using modern data science technology. I can promise you that will be an interesting discussion too!

Day 2 problems in the Cloud

When moving to the Cloud companies run into “Day 2 problems”. The technology, interfaces and how to use all the stuff are super easy however on the second day comes the real challenge: How to use all this great stuff in a structured way on company/enterprise level?

Here is a great read on “Day 2 problems in the Cloud“. Day 2 is a management problem, not a technology problem. Adoption and implementation require different skill sets, than innovation. These have to be put to work.

Two interesting reads on team structure and focus

(Re)discovered two interesting reads on team structure and focus this week that I like to share.

Pioneers, Settlers and Town Planners

pioneer, settler and town planner The Pioneers, Settlers and Town Planners approach help structure organisations and teams. It takes into account how activities and practices move from chaotic (poorly understood, uncertain, constantly changing, rare, future source of worth) to more linear (well-defined, predictable, stable, common, cost of doing business) and how organisations contain a mass of these activities and practices. Understanding this and using the right methods and tactics is important to creating a balance between the unstable but potentially high margin activities (chaotic) and the stable and low margin (linear).

McKinsey’s Three Horizons of Growth

McKinsey’s Three Horizons of Growth are all about keeping you and your teams focused on growth and innovation. This strategy framework requires you to categorize your goals into 3 different ‘horizons’. You can also split the activities of your team to the three types of activities that are related to fulfilling the goals for the different horizons. In that case, a team could spend (as a rule of thumb):

  • 70% on activities that are most closely aligned to your current business.
  • 20% on taking what you already have, and extending it into new areas of revenue-driving activity.
  • 10% on introducing entirely new elements to your business that don’t exist today.

This helps ensure that you consistently balance your focus between the needs of today (horizon 1), the future state of your business (horizon 3) and the steps that you need to take to get there (horizon 2).

Travis Law

Travis’s Law:

Our product is so superior to the status quo that if we give people the opportunity to try it, they will defend it and demand its right to exist.

Travis Law is a quote from a speech by Travis Kalanick founder of Red Swoosh and wider known Uber. In his CEO role at Uber, there have been quite some scrimmages with local governments and unions. This law is one of the strategies what Uber uses to influence the outcomes of these discussions. They have been pretty successful with it.

Will it work outside Uber?

There is one other company that comes to mind that faces similar discussions with unions and governments that has used this approach successfully: Airbnb.

Some background

In a Havard Business Review article on Uber (and it’s assumed illegality) it is stated:

What’s more, Uber’s most distinctive capabilities focused on defending its illegality. Uber built up staff, procedures, and software systems whose purpose was to enable and mobilise passengers and drivers to lobby regulators and legislators — creating political disaster for anyone who questioned Uber’s approach.

Related laws

In a way, this is an extension to what Peter Thiel stated in Zero to One:

As a rule of thumb proprietary technology must be at least 10 times better than its closest substitute in some important dimension.

Where Thiel focusses on the product development part, Travis goes from the demand. The demand from users that is used to overcome obstacles, like protectionist rules and laws and unions acting to protect jobs and actually sometimes more investments and lobbies by unions.

Presentation Applying “web scale” patterns in the bol.com back office

Codemotion web scale pattern presentationThis week at Codemotion I gave a presentation on web scale patterns and how we apply them in the bol.com back office services. Codemotion is the biggest tech conference in Italy and one of the most important in Europe.

The presentation shows how we go from business goals to software patterns. The following patterns were covered: CQRS, event sourcing, polyglot persistence and micro services.


Presenting at Codemotion next week on web scale patterns

Peter Paul van de Beek CodemotionNext week I’ll be presenting in Amsterdam at Codemotion on web scale patterns in bol.com back office services. Codemotion is the biggest tech conference in Italy and one of the most important in Europe, with a network of more than 30k developers.

The title of my presentation is: “Applying “web scale” patterns in the bol.com back office”. In the session, we show you how we use “web scale” patterns to achieve scalability and flexibility in our back office software. We will guide you through how we apply patterns like CQRS, event sourcing, polyglot persistence and micro services to solve puzzles in our back office services. Interesting is that in our experience we don’t need these just to solve a technical problem. It helps us to solve some of our business problems!

See you on May 16 and 17 in Amsterdam. There is a short interview with me on Codemotion world.

web scale patterns in the bol.com back office – Mixed SQL – NoSQL

In the previous weeks, we started a series of blog posts that show you how we use “web scale” patterns to achieve scalability and flexibility in our back office software. The previous patterns discussed were Event Sourcing and CQRS. This week we will dive into mixed SQL – NoSQL. Showing you how this doesn’t just solve a technical problem, they help us solve our business problems!

Mixed SQL – NoSQL

Where needed in our services we are moving away from pure SQL. We create a mix with other types of storage. So we are using NoSQL (Not only SQL). One could also call this polyglot persistence. The notion that your application can write to or query multiple databases or one database with multiple models. It uses the same idea as polyglot programming. This expresses the idea that applications should be written in a mix of languages to take advantage of the fact that different languages are suitable for tackling different problems.

RTN – Billing platform for our retailers

RTN stores all kinds of transactions to charge and pay partners in our LvB (Fulfilment by bol.com) operations. In this part of our operation, we store and fulfil products in our warehouses for retailers that sell their goods on our platform.

The transactions to create invoices for our LvB partners stems from a number of services. We have all kinds of different attributes we want to account for to know why decisions have been made and for auditing purposes. These attributes depend on the transaction type. It was decided that the attributes wouldn’t be a part of the transactions table since they are only filled for a part of the records.

To accommodate for the attributes that depend on the transaction type we created an additional column in the table that is able to store key-value pairs in JSON format. The use of a pure SQL solution would have resulted in a weak design. As would the use of a pure NoSQL. In these cases, they work great together.

FNK – Warehouse orders

FNK processes our customer orders to create warehouse orders. It determines the warehouse that will fulfil the customer demand and instructs the warehouse to fulfil the order. Besides regular warehouses, it also communicates with our warehouse for digital products (e-books and software downloads) and with retailers that sell their products on our platform and take care of the fulfilment themselves.

These retailers have requirements that differ from the other warehouses. To accommodate these while avoiding a to a specific part in this services we introduced an additional column that stores XML. This mixture of SQL (one table for all warehouse orders) and NoSQL (stored XML) results in a simple model that can handle requirements that are only needed for a part of the orders. Since the data in the XML is hardly needed in this service but mostly in downstream services, there are no drawbacks on performance.

What we learned

The NoSQL parts in these mixed data stores are mostly used to read from. If you need to specifically filter on these of have a requirement to use them in joins performance will degrade.

Next in web scale patterns in the bol.com back office

In the next week’s episode on the following subject will be published:

  • Micro services