Tag Archives: performance

web scale patterns in the bol.com back office

web scale patterns in the bol.com back office – Event Sourcing

Last week we started a series of blog posts we show you how we use “web scale” patterns to achieve scalability and flexibility in our back office software. Last week’s pattern we discussed was CQRS. This week we will dive into Event Sourcing. Showing you how this doesn’t just solve a technical problem, they help us solve our business problems!

Event Sourcing

The idea of Event Sourcing is that every change to the state of a system is captured in sequence and that these events can be used to determine the current state. Consequently, the state of the system for any point in time can be determined by replaying the events. The structure of the service changes from storing state to storing events.

The most obvious that we gain by using Event Sourcing is that we have a log of all the changes. We can see everything that happened. This enables us to:

  • Do a complete rebuild;
  • Determine the state of the system at any point in time;
  • Event replay – Compute the consequences of a change in a past event of recalculate the consecutive states based on the proper sequence of events (in case messages in an asynchronous communication weren’t received in the proper order).

Using Event Sourcing can feel a little bit awkward for some developers. However, it offers a variety of opportunities. One could replay the events on a test environment to see exactly what happened on pro, while you have the ability to stop, rewind and replay the events running a debugger. This provides also a way to do parallel testing before promoting an upgrade to production.

Where do we use it at bol.com?

web scale patterns in the bol.com back officeOne of the examples where we use Event Sourcing is Condition Management and especially the calculations of accruals and invoices for (purchasing) conditions. A large set of our purchasing conditions is based on either purchasing amounts or values and sales amounts and values. In general these purchasing conditions have to attributed to (sets of) single products, product categories, suppliers and brands.

Storing the events that represent the purchase and sales of goods allows us to implement functionality that would be very hard to develop if we wouldn’t. Typically a purchasing condition isn’t agreed with a supplier of a brand on the first of January. While it could be valid from the first of January. The Event Sourcing model allows us to handle conditions that are entered into the system somewhere in March or April that are valid from the first of January. These conditions will be handled by passing all the events from the start date and the appropriate accruals and invoice can be created.

With the Event Sourcing model, we are also more loosely coupled to the source services for purchasing and sales. Our calculations can handle events that are captured out of sequence or even very late. Condition values are still calculated properly and handled as accounting and controlling have prescribed.
For the future, we are planning to implement scenario run through and comparisons. This would support our buyers while negotiating with suppliers.

Next in web scale patterns in the bol.com back office

In the next week’s episodes on the following subjects will be published:

Performance testing at bol.com

On Monday October 12th Rob de Groot and Chris Kramer of the bol.com test automation team and development process innovation team will present on performance testing at bol.com. The presentation is for the Dutch Web Performance & Operations Meetup, as a pre-event for the WebPerfDays.

Rob and Chris will elaborate on how performance testing evolved at bol.com and show how modern tooling like Docker and Mayfly fits into the future of performance testing? Mayfly is the user story centric Continuous Delivery development platform, developed by bol.com.

The event will be hosted at bol.com headquarters in Utrecht (map).

RSVP and join!

VirtualBox improve virtual machine performance

VirtualBox improve virtual machine performanceVirtual Machines are a great way to test stuff or use valuable tools in a separate environment. Think of test driving an OS, like Ubuntu 12.04 or Windows 8. Other examples include using pre-build appliances to use Oracle SOA Suite and BPM or WebLogic Server and Java tools.

Improve VirtualBox appliance performance

These Virtual Machines can be quite demanding for resources on your PC or laptop. Fortunately there are ways to speed up VirtualBox appliances. Besides increasing the allocated CPU power and RAM there are a few less expected things you can do:

  • Create fixed size disks – A preallocated disk will have less fragmentation. Adding of files to the virtual disk will be faster (spaces is already reserved). The downside is that a fixed-size disk uses more space on your hard disk.
  • Exclude the Virtual Machine directory from the virus scanner – Scanning there from the host isn’t very useful either. So add this directory to the exclusion list.
  • Put the VirtualBox files on non-system disk – Put the VirtualBox files on a non-system disk. The virtual disk and your hosts’s OS won’t be competing to read from or write to these same disk.
  • Make shure the Intel VT-x and AMD-V setting is ON – These are processor extensions that improve virtualization performance.

Oracle ESB performance enhancements

Because the growth in the number of messages that one of our customers is receiving on it’s Oracle ESB, they were looking into how to enhance it’s performance. Oracle came up with the following clues:

  • Upgrade SOA Suite to a higher patch level
  • Reduce logging (including instance tracking in ESB)
  • Increase the number of listeners in ESB on Queues
  • Turn off payload validation

In order to see what these could do for them we did some loadtesting. The upgrade to a higher patch level of SOA Suite was needed at least to get the ESB working correctly with the configured number of listeners. However because of some issues running Oracle Universal Installer there wasn’t any time left to verify that the patch resulted in increased performance.
For functional and application management reasons turning of logging was not considered an option at this stage.

Increase the number of listeners in ESB on Queues

set ESB listeners in console

set ESB listeners in console

For each ESB System the number of listeners can be configured. These listeners are used to read from JMS topics. If there is only one listener (which is the default) message are read from the JMS Topic one at a time. The first screenshot shows where to set the number of listereners using the ESB Console.

Average response time

Average response time

To determine the optimum number of listeners for the specific ESB, OS, hardware combination we did some loadtesting. The results for the average response time are shown in the next screenshot.

As expected the respons time increases at a certain point when the number of listeners is raised.

With 90% and max response time

With 90% and max response time

“Time out” messages are never created by averages. While this first graph looks great and is very convincing, we are even more interested in the 90% and maximum respons times. A chart for these statistics has been included as well. Analyzing this graph one can conclude that this has an even stronger support to set the number of listeners to an appropriate level.

From this experiments it can be concluded that the response time can significantly be reduced by setting the number of listeners of an ESB system to the appropriate level. The maximum respons time has can be decreased by over 50%. This has a significant impact on the availability of the services exposed using the ESB.

Turn off payload validation

Turn off payload validation

Turn off payload validation

Another idea to enhance the performance is to turn off payload validation. Needless to say that this also has functional aspect. In most cases the validation that is performed here has some kind of business (related goal), like data quality. To switch it off, has impact, and the validation has te be migrated to some place else in the software.
Response time

Response time

The performance results can be dramatic. In a positive way! As is shown in the figure on the right, the respons time can be reduced to a fifth of the original. This goes for both the average and the maximum respons times.

If you can change your service so payload validation at this point is no longer required, this is a great step in enhancing the Oracle ESB’s performance.