Net als de afgelopen jaren heb ik afgelopen vrijdag een presentatie over Agile Architectuur gegeven. Kern van de presentatie dit jaar is dat bij het schalen van Agile vooral de problemen/uitdagingen die je onderweg tegenkomt aangepakt moeten worden en niet klakkeloos een framework gaat implementeren. Aan de hand van voorbeelden leg ik uit hoe we dit binnen bol.com aanpakken.
On Thursday, May 12, 2016 I will be presenting on pragmatic microservices at the GOTO Night organised at bol.com. The presentation will be the support act for Randy Shoup. Check some of his previous presentations on SlideShare.
We have been around in e-commerce for years. However compared to other companies we’re young. Some would say we are in the scale up phase. In a number of ways we are experiencing a rapid growth. What does our IT need to stay innovative and scale to enable all this? What are the tradeoffs that are made for innovation in IT?
This year we won the Best Web Shop award because of our “efforts to get the difficult to achieve basics right that make the difference for customers”. IT has a large role in achieving this, at the scale of a web shop like bol.com. Did (micro)service make the difference to achieve this?
At bol.com we have a pragmatic, business value driven approach to (micro)services. In this presentation we share insights and the tradeoffs we made so IT enables to scale and innovate.
Presentation Pragmatic (Micro)Services
Here is the presentation I used:
While preparing guidelines for the usage of the Oracle Service Bus (OSB) I was looking for a definition of a Service Bus. There wasn’t one on my blog yet (more posts on integration) so i decided to use the following and share them with you.
Forrester Service Bus definition
From 2009 Forrester has used this one:
An intermediary that provides core functions to makes a set of reusable services widely available, plus extended functions that simplify the use of the ESB in a real-world IT environment.
Erl Service Bus definition
Thomas Erl offers the following description of a Service Bus::
An enterprise service bus represents an environment designed to foster sophisticated interconnectivity between services. It establishes an intermediate layer of processing that can help overcome common problems associated with reliability, scalability, and communications disparity.
An Enterprise Service Bus is seen by Erl et al as a pattern. That is why it is even more important to share what that patterns is. Later on I’ll also shortly describe the VETRO pattern. Also a very useful pattern to use when comparing integration tools or developing guide lines.
Erl Enterprise Service Bus pattern
On the SOA patterns site we learn that an enterprise service bus represents an environment designed to foster sophisticated interconnectivity between services. The Enterprise Service Bus pattern is a composite pattern based on:
- Asynchronous Queuing basically an intermediary buffer, allowing service and consumers to process messages independently by remaining temporally decoupled.
- Service Broker composed of the following patterns
- – Data Model transformation to convert data between disparate schema structures.
- – Data Format transformation to dynamically translate one data format into another.
- – Protocol bridging to enable communication between different communication protocols by dynamically converting one protocol to another at runtime.
- Intermediate routing meaning message paths can be dynamically determined through the use of intermediary routing logic.
- With optional the following patterns: Reliable Messaging, Policy Centralization, Rules Centralization, and Event-Driven Messaging. Also have a look at slide 12 etc of the SOA Symposium Service Bus presentation.
VETRO pattern for Service Bus
The VETRO pattern was introduced by David Chappell, writer of the 2004 book Enterprise Service Bus.
- V – Validate: Validation of messages eg based on XSD or schematron.
- E – Enrich: Adding data from applications the message doesn’t originate from.
- T – Transform: Transform the data model, data format or the protocol used to send the message.
- R – Routing: Determine at runtime where to send the message to.
- E – Execute: You can see this as calling the implementation.
We also used this pattern to compare Oracle integration tools and infrastructure. It can be very well used while choosing the appropriate tools for a job and deciding on guidelines on how to use these tools.
The world’s largest conference dedicated to SOA, cloud computing and service technology will have it’s 2012 version in London! Hosting the 5th SOA Symposium and the 4th International Cloud Computing Symposium on September 24-25. This brings the symposium back to Europe after last years visit to Brasilia, Brazil. The SOA Symposiums website has been rebranded to Service Tech Symposium.
There are several blog posts on previous editions of the SOA Symposium available in blogs. During this years event the following books will be launched:
- Cloud Computing: Concepts & Technology
- SOA with REST: Principles, Patterns & Constraints
- Next Generation SOA: A Real-World Guide to Modern Service-Oriented Computing
Call for presentations
The 2012 program committee invites submissions on all topics related to SOA, cloud computing and service technologies. The primary tracks are:
- Cloud Computing Architecture & Patterns
- New SOA & Service-Orientation Practices & Models
- Service Modeling & Analysis Techniques
- Service Infrastructure & Virtualisation
- Cloud-based Enterprise Architecture
- Real World Case Studies
- Service Engineering & Service Programming Techniques
- Interactive Services & the Human Factor
- New REST & Web Services Tools & Techniques
On December 13th Whitehorses will host a seminar on SOA and Governance. During this seminar we will show the value of a proper architecture and governance for your organization. In the presentation you will get clear guidelines and steps on a pragmatic approach for implementing a manageable SOA solution.
Some of the topics:
- What is SOA Governance and Why do we need it?
- SOA reference architecture – The importance of solid standardization.
- Service life-cycle governance – Design and build the right services and the proper way to reuse them.
- Service repository – With examples of repositories based on Oracle Enterprise Repository (OER) and a wiki.
While in a previous post I was arguing that we should handle industry models with care, because of very inconvenient side effects. This week I’ll blog in a similar way on centralization. Among the effects of centralization are often overlooked or neglected dis-economies of scale.
Dis-economies of scale
One of the main reasons for centralization is to gain economies of scale. Less known are the dis-economies of scale. I’ll give some examples in the paragraphs below.
The cost of communication between the central group and the rest of the organization. Although there are lots of tools that make communication easier. Distance in the physical sense or within an organization can create boundaries. These have to be dealt with and there are costs incurred for that. Besides that it has to be clear who to communicate for what matters. This, in my experience, is not always the case. With a greater (organizational) distance more effort has to be put into this.
There is a large possibility that top heavy management in a centralized department becomes isolated from the effects of their decisions. In other words the feedback loop is broken. Because the communication loop is broken, decision become more and more dysfunctional. This due to the lack of real world knowledge that should be incorporated in these decisions.
Centralization can lead to reduced agility. On one hand standardization is a great asset. The larger part of architecture, whether it is enterprise architecture, process architecture or infrastructure architecture, is about standards and reducing the “solution space”. This has several advantages, among which the reduction of software- and systems entropy. The downside of a centralized body that maintains standards is that it probably will lead to inertia and unwillingness to change.
I’m a big fan of (open) standards. They simplify life! However we should not neglect that standardization comes at a cost. There are the costs for implementing, adapting to and maintaining standards in our organization. Say for example that we use a canonical (data) model. There is are maintenance costs (at least some effort) while adopting to change outside and within our organization. These costs of standardization tend to be hidden.
What to do?
Bring the effects described before into the business case for centralization. You did make sure that there was some sort of trade off when you decided to centralize a certain part of your organization didn’t you?
Take measures to prevent these risks. It goes without saying that these measures will take effort, time and possibly money. Now you know you’re going to take measures don’t you?
Most discussions on Cloud Computing I’ve been reading are focused on the infrastructure and technology part. It offers easy to deploy infrastructures or even applications in a very scalable way. All this in a pay-per-* way. And here is where a CFO should get interested. Moving to a Cloud implies moving from CAPEX to OPEX. Usually a CFO has an idea on how to keep these balanced. The Enterprise Architecture of some organizations even have very strict guidelines on whether certain expenses should the one or the other. So that’s the first one to thing about…
As was stated in a previous blogpost on measuring the business value of SOA, project metrics for business value created by SOA projects, IT projects, or even projects in general are rare. If I were a CFO this would worry me.
Besides that SOA efforts in a way also demand a different way of cost accounting than the traditional silo based. If my organizational unit owned (and had to account for the costs) of a rather popular often reused service, I would like to charge them. Say for example in a pay-per-service-call way. How do the financial systems under the responsibility of the CFO facilitate this?
Of course there will be lots of other stuff on your agenda if you are the CFO. But hey due to the crisis interest rates are low, labor is cheap, as are materials, so why not invest now in the foundation/infrastructure for the future 😉
If you’re a CFO and – by incident – are reading this blogpost please let me know what you think, and add a comment…
As Anne Thomas Manes stated in her presentation on Measuring the Business Value of SOA a 2009 Gartner study showed that
- 36% of SOA projects lack a business justification
- 1% of all SOA efforts actually measured benefits
From these statistics it doesn’t seem to be a natural thing to do, measuring the business value of SOA. By measuring the business values we mean value in monetary terms – “hard cash”. Think in terms like:
- Increased revenue
- Lower costs
- Better use of assets
- Solve customer business needs
In order to have a good assessment of these we need a solid baseline measurement. This is where it gets hard. How many organizations actually have baselines like these. So the intricacies of measuring the business value of SOA aren’t necessarily related to SOA! It might very well be an issue for IT or even businesses in general.
What is specific for SOA or any other architecture or maybe project management approach, is to single out SOA as the direct responsible mechanism for the business value created (causality). In other words what part is SOA specific and what is “just” the availability of a new application? SOA has an indirect impact on business outcomes.
As an example of the previous point: In one of the SOA efforts that I have been involved in – that was at least in my opinion successful – the realization of the business case was very clear: 5 to 10 employees of a department had no longer work to do after 11.00 am. However because we automated a process that wasn’t automated before, it would be very hard for me to point out what part of the savings was actually directly caused by implementing in an SOA context.
This is one of the reasons why I’ve been advising organizations to implement SOA as part of their “normal” projects, or at least projects with a valid business case. Dealing with your project in a SOA way will not only deliver the monetary outcome described in the business case. Applying SOA principles like “Separation of concerns” and “Loose Coupling” will yield in solutions that are modular, interoperable, and shareable.
The agenda for the SOA Symposium 2010 has been posted. Again there are very interesting sessions during this 2 day conference. The largest and most comprehensive in the field of SOA and Cloud Computing. The Real World SOA Case Studies track offers a great opportunity to learn from the experience of others. In this track you will find:
Real-life accounts of successful and failed SOA projects discussed first-hand by those that experienced the project lifecycles and have a story to tell. These veteran practitioners will provide advice and insights regarding challenges, pitfalls, proven practices, and general project information that demonstrates the intricacies of implementing and governing service-oriented solutions in the real world.
I will be presenting the first session in this track on Using a Service Bus to Connect the Supply Chain. If you have any topics or questions in advance that you think I should address, please post them in the comments. Hope to meet you in Berlin.
On October 5 and 6 2010 the worlds largest SOA and Cloud Computing event will be held in Berlin; the SOA Symposium. The International SOA and Cloud Symposium brings together lessons learned and emerging topics from SOA and Cloud projects, practitioners and experts.
There is a call for presentations:
The SOA and Cloud Symposium 2010 program committees invite submissions on all topics related to SOA and Cloud, including but not limited to those listed in the preceding track descriptions. While contributions from consultants and vendors are appreciated, product demonstrations or vendor showcases will not be accepted.
All submissions must be received no later than June 30, 2010. An overview of the tracks can be found here. Other resources: