Tag Archives: DevOps

Velocity 2015 Amsterdam

Thursday was a very interesting day for me at Velocity 2015 Amsterdam, build resilient systems at scale. It is one of the best conferences I attended in the last years. Using some quote’s and bullets I’ll give a little insight.

On retro’s, post mortems, etc

Lindsay Holmwood showed that what goes wrong in retrospectives, post mortems and the like is mostly based on:

  • Confirmation bias – aka ignore alternatives
  • Hindsight bias – aka – alter memories to fit a narrative. Talk about events with the knowledge of the outcome.

To overcome these we could use techniques like: Take opposing viewpoints (on purpose, to investigate things), contrarian thinking, let people explain stuff in terms of foresight and all kinds of sharing information.
In short for this to work we need a safe environment where people can speak up. Starting from the believe that everyone did the best they could. And always keep in mind that there is a difference between work as imagined and works as done.

Optimizing teams in a distributed world – Conway’s 3 other laws

Conway’s Law stems fro the greater part from his 1967 paper: How do committees invent.
The slides and all references mentioned in the presentation.

  1. Whose structure is a copy of the organisation’s structures. To put it different: Communication dictates design. Also check The Mythical Man Month and Dunbar’s number. So manage communication between teams.
  2. – Doing it over – There is never enough time to do something right, but there is always enough time to do it over. Engineering and architecture are always about: Trade offs. Also check: Satisfying vs Sacrificing. So remember it is a continuous process.
  3. If you or your team cannot explain all the code in your release package your release is too large.

  4. Homomorphism – If you have 4 groups working on a compiler, you’ll get a 4-pass compiler. So organise teams in order to achieve what you want (around business capabilities).
  5. Disintegration – The bigger they are, the harder they fall. Time is against large projects and teams. Aim for a scope that supports a release cycle of two weeks or less. So keep your teams as small as necessary.

It is better to be too small than too big.

We are all DevOps

One of the best talks on DevOps in the Etsy world by Katherine Daniels.

On hiring:

It is easier to teach someone a new technology skill, than to teach someone not to be an asshole.

Continuous Delivery at bol.com

Last month two of our software engineers Mihaela Tunaru and Mary Gouseti were invited to give a presentation of how continuous delivery is done at bol.com. The presentation gives a good insight in the state of continuous delivery at bol.com from a software engineering perspective.

In case you want to know more from the operations perspective check Mayfly on GitHub and the presentation below. Maarten Dirkse gave a talk Docker your user stories using Mayfly.

Mayfly is a development platform built by bol.com. Mayfly speeds up your service development by wrapping your scrum user story code in containers, testing it in an isolated, production-like environment and automatically enforcing your Definition of Done.

Book – The Phoenix Project

Book The Phoenix ProjectThe Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win is written the by Gene Kim in the tradition of The Goal (1984, by Dr. Eliyahu M. Goldratt). The Goal is a management novel explaining the Theory of Constraints. This book, The Phoenix Project shows how the theory in The Goal works in an IT environment.

The Goal – Theory of Constraints

In simple terms the Theory of Constraints is about:

A chain is as strong as its weakest link.

In this theory the first step is to identify the constraint. Step 2 is to exploit the constraint. In other words, make sure that the constraint is not allowed to waste any time. Only by increasing flow through the constraint can overall throughput be increased. This to the extend that improving something anywhere not at the constraint is an illusion.

Because of the need for flow, work in process (WIP) is the silent killer. Therefore, one of the most critical mechanisms in the management of any plant is job and materials release. Without it, you can’t control WIP.

The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win

The Phoenix Project describes the problems that almost every IT organization faces, and then shows the practices (based on the Theory of Constraint, Lean and more) of how to solve the problems. The main character Bill, is thought how to deal with these problems using the Socratic Method. Each dialogue a question is posed to which in turn causes Bill to think and to talk to his colleagues to come up with a solution to their problem.

Bill starts to see that IT work has more in common with manufacturing plant work than he ever imagined. Leading to the application of the Theory of Constraints in terms like:

The First Way helps us understand how to create fast flow of work as it moves from Development into IT Operations, because that’s what’s between the business and the customer. The Second Way shows us how to shorten and amplify feedback loops, so we can fix quality at the source and avoid rework. And the Third Way shows us how to create a culture that simultaneously fosters experimentation, learning from failure, and understanding that repetition and practice are the prerequisites to mastery.

Work in process in IT perspective

Until code is in production, no value is actually being generated. It’s merely WIP stuck in the system. By reducing the batch size, you enable a faster feature flow. In part this is done by ensuring the proper environments are always available when they are needed. Another part is automating the build and deployment process. Here we recognize that infrastructure can be treated as code, just like the application that Development ships. This can enabled to create a one-step deploy procedure.

Besides the parts mentioned before this requires removing a unneeded (since no value is created) hand off between Development and Operations. For this to work the two have to be integrated, not separated.

Like in a manufacturing plant, in IT, it is crucial to manage the release of work to the shop floor / development and to track the work in process. There are a lot of visual aids available to support this, like Kanban or scrum boards. All have their origin in lean or agile ways of working.

No need to say that in the novel this all works out pretty well 😉 In real life we see that these principles work, however more iterations are needed to really improve things. These iterations at first look like failures because of the acceleration of entropy. They are needed in the learning process of people and organization. Reduce the feedback cycle and learn fast!

On the relation between business and IT

There are some interesting statements in the book, that are heard more often in the industry.

IT is not just a department. IT is a competency that we need to gain as an entire company.
We expect everyone we hire to have some mastery of IT. Understanding what technology can and can’t do has become a core competency that every part of this business must have. If any of my business managers are leading a team or a project without that skill, they will fail.

And:

In ten years, I’m certain every COO worth their salt will have come from IT. Any COO who doesn’t intimately understand the IT systems that actually run the business is just an empty suit, relying on someone else to do their job.

Personally i think they hold at least some value. Please share your ideas in the comments.