Tag Archives: Lean

Lean

Agile and lean reading list summer 2015

These are the books on agile and lean on my reading list for this summer:

Lean Enterprise

lean enterpriseThe book of the moment on innovation at scale. According to wikipedia (Lean Enterprise):

Lean enterprise is a practice focused on value creation for the end customer with minimal waste and processes. The term has historically been associated with lean manufacturing and Six Sigma due to lean principles being popularized by Toyota in the automobile manufacturing industry (TPS) and subsequently the electronics and internet software industries.

The publishers site on Lean Enterprise:

Through case studies, you’ll learn how successful enterprises have rethought everything from governance and financial management to systems architecture and organizational culture in the pursuit of radically improved performance. Adopting Lean will take time and commitment, but it’s vital for harnessing the cultural and technical forces that are accelerating the rate of innovation.

Agile IT Organization Design

Agile organisation designIn order to scale Agile, it is not enough to just replicate development practices and techniques across teams. We also need to review organization structure and management controls to see if they are in tune with what is needed for responsive IT. Unless we do so, overall IT performance is unlikely to improve. This highly recommended book provides a basis for reviewing and reshaping the IT organization to equip it better for the digital age.

Being Agile in Business

Agile in businessAgile and lean are fast and efficient methodologies you need to change the way you work. This book introduces you to the essential skills and mindsets of agile and lean and quickly encourage you to start thinking differently.

Sometimes development is just work

No matter how cool your job is, no matter how many people are looking at you or your company for best practises, sometimes developing software is just work 😉 On this blog I’ve shared examples of companies that people nowadays see as successful, like Netflix, Twitter, Spotify, or the online retailer bol.com.

To prove my point I’ve checked the release notes of Netflix and Spotify apps. Here is what they show for recent updates:

Software development at Spotify is just work

You can find recent release notes for Spotify. For future reference here is a screenshot of how these looked today:
Software development at spotify is just work

As you can see it is mainly fixes and a new translation… Where did all the fun stuff go. Think the cat took it? So crafting software could be “just” improving and step by step creating a great product!?

Software development at Netflix is just work

Now lets look at Netflix. Just looked up the release notes of Netflix in the iTunes store. Here is how they looked today:
Software development at Netflix is just work

Wow! Updates and bug fixes. That sounds really cool. That must be loads of fun. So could it be that even working on awesome apps for great companies is (at least for a part) just work?

Success needs work

So sometimes software development is just work. Just don’t forget:
The only time success comes before work is in the dictionary.
Could have said it better Harvey: The only time success comes before work is in the dictionary.
Fun and play are a part of you as a person. Work is just a way to make it flow…

Book – The Phoenix Project

Book The Phoenix ProjectThe Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win is written the by Gene Kim in the tradition of The Goal (1984, by Dr. Eliyahu M. Goldratt). The Goal is a management novel explaining the Theory of Constraints. This book, The Phoenix Project shows how the theory in The Goal works in an IT environment.

The Goal – Theory of Constraints

In simple terms the Theory of Constraints is about:

A chain is as strong as its weakest link.

In this theory the first step is to identify the constraint. Step 2 is to exploit the constraint. In other words, make sure that the constraint is not allowed to waste any time. Only by increasing flow through the constraint can overall throughput be increased. This to the extend that improving something anywhere not at the constraint is an illusion.

Because of the need for flow, work in process (WIP) is the silent killer. Therefore, one of the most critical mechanisms in the management of any plant is job and materials release. Without it, you can’t control WIP.

The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win

The Phoenix Project describes the problems that almost every IT organization faces, and then shows the practices (based on the Theory of Constraint, Lean and more) of how to solve the problems. The main character Bill, is thought how to deal with these problems using the Socratic Method. Each dialogue a question is posed to which in turn causes Bill to think and to talk to his colleagues to come up with a solution to their problem.

Bill starts to see that IT work has more in common with manufacturing plant work than he ever imagined. Leading to the application of the Theory of Constraints in terms like:

The First Way helps us understand how to create fast flow of work as it moves from Development into IT Operations, because that’s what’s between the business and the customer. The Second Way shows us how to shorten and amplify feedback loops, so we can fix quality at the source and avoid rework. And the Third Way shows us how to create a culture that simultaneously fosters experimentation, learning from failure, and understanding that repetition and practice are the prerequisites to mastery.

Work in process in IT perspective

Until code is in production, no value is actually being generated. It’s merely WIP stuck in the system. By reducing the batch size, you enable a faster feature flow. In part this is done by ensuring the proper environments are always available when they are needed. Another part is automating the build and deployment process. Here we recognize that infrastructure can be treated as code, just like the application that Development ships. This can enabled to create a one-step deploy procedure.

Besides the parts mentioned before this requires removing a unneeded (since no value is created) hand off between Development and Operations. For this to work the two have to be integrated, not separated.

Like in a manufacturing plant, in IT, it is crucial to manage the release of work to the shop floor / development and to track the work in process. There are a lot of visual aids available to support this, like Kanban or scrum boards. All have their origin in lean or agile ways of working.

No need to say that in the novel this all works out pretty well 😉 In real life we see that these principles work, however more iterations are needed to really improve things. These iterations at first look like failures because of the acceleration of entropy. They are needed in the learning process of people and organization. Reduce the feedback cycle and learn fast!

On the relation between business and IT

There are some interesting statements in the book, that are heard more often in the industry.

IT is not just a department. IT is a competency that we need to gain as an entire company.
We expect everyone we hire to have some mastery of IT. Understanding what technology can and can’t do has become a core competency that every part of this business must have. If any of my business managers are leading a team or a project without that skill, they will fail.

And:

In ten years, I’m certain every COO worth their salt will have come from IT. Any COO who doesn’t intimately understand the IT systems that actually run the business is just an empty suit, relying on someone else to do their job.

Personally i think they hold at least some value. Please share your ideas in the comments.

Lessons of product development at Netflix

Just a month after sharing my post on Spotify engineering culture, I found a post on Startup lessons from Netflix. That was written inspired by a talk on fast delivery devops by Adrian Cockcroft. Who spent a long time building up Netflix’s cloud infrastructure and spearheaded the development of many new cloud-related technologies and techniques at the company.

Adrian Cockcroft’s lessons of product development at Netflix

Adrian’s lessons of product development at Netflix are summarised in this sheet:
Lessons of product development at Netflix
Besides from the different angle and focus on cloud, I think that there is quite some overlap with the Spotify presentations. If you have a different take at this, please leave a comment or meet me at the LAC congres where I will be presenting on time-to-market vs architecture…

Spotify engineering culture part I & II

I’ve been reading quite some article on engineering culture and ways of working. The videos on Spotify Labs are among the best sources I’ve watched or read in the last year on the subjects of agile and culture. Recently the second part of their series on Spotify’s engineering culture was released.

Spotify engineering culture part I


Important take aways for me were:

  • Agile over scrum
  • Principles over practices
  • Servant over master

Spotify engineering culture part II

Very cool that one of my favorite quotes by Mario Andretti was used in the video:

If everything seems under control, you’re not going fast enough.

To cope with this aspect you need a fail friendly environment and a limited blast radius. For the first focus on fail recovery instead of an fail avoidance. For the latter focus on a decoupled architecture.

A healthy culture heals broken processes! Growing organizations have growing pains. Culture can either magnify or heal them.

Update: Henrik Kniberg on Scaling agile at Sporify

The hour talk that Henrik Kniberg gave on Scaling agile @ Spotify is also available on vimeo:

.

New Whitebook article (Dutch) – Integratie maar dan lean

lean integrationRecently my latest Whitebook was published, called ‘Integratie maar dan lean‘. The article focusses on solid tips to move your integration practise to a more lean version.

It’s in Dutch. International readers can use Google Translate. An English translation can be provided on request. Please leave a comment with your email. Have your integration lean 😉

Export Oracle BPM metrics to a data warehouse

Managers and teamleads rely on measurements to know how processes and teams are performing. With an increased BPM effort and automated support of processes, more and more questions on solid information from these systems rise. Oracle BPM offers an easy way to export metrics to a data warehouse. Oracle BPM captures standard and business specific metrics and exposes these through a BPM process star schema.

Export Oracle BPM metrics to a data warehouse

To export Oracle BPM metrics to a data warehouse the SOAINFRA schema in which all configuration and (runtime) instance data is stored offers a number fact tables. These standard views enable the extraction of information for BI usage. For Oracle BPM release prior to PS4FP (11.1.1.5.1) the views have to be created manually. From patchset 4 onwards the process has been automated. Please note that it is not recommended to run the BI reports on the BPM process database.

An overview of the standard fact tables:

  • BPM_PROCESS_PERFORMANCE_V – offers standard metrics (like start and end time and running time in seconds) for completed processes.
  • BPM_ACTIVITY_PERFORMANCE_V – offers standard metrics for completed activities, completed intervals, measurement marks and counters for both in-flight and completed process instances.
  • BPM_PROCESS_INSTANCE_V – offers standard metrics for in-flight process instances. Because of that the information is only relevant at the time the view is queried. When processes move forward the information in this view refelcts the progress.
  • BPM_ACTIVITY_INSTANCE_V – offers standard metrics for in-flight activities and interval instances.

An overview of the dimension tables:

  • BPM_PROCESS_DEFINITION_V – including data on domain, composite, label and revision.
  • BPM_ACTIVITY_DEFINITION_V – including the type of activity: UserTask, Gateway, Event, Measurement interval, etc.
  • BPM_ROLE_DEFINITION_V – also allows you to see whether the role is the process owner.
  • CUBE_INSTANCE
  • COMPOSITE_INSTANCE

Example queries of BPM process metrics

Average process running time by process:

SELECT process_name
,      avg(process_running_time_in_msec)
FROM   bpm_process_performance_v 
GROUP  BY process_name
;

The number of faults by process:

SELECT process_name
,      COUNT(sequence_id)
FROM   bpm_process_performance_v
WHERE  process_discriminator = 'instance_system_fault'
GROUP BY process_name
;

When you run the process in multiple domains on the same server join with the BPM_PROCESS_DEFINITION_V and differentiate on DOMAINNAME.

Example queries of task performance metrics

Average, minimum and maximum time taken by a participant in a process per activity:

SELECT process_name
,      activity_label
,      role_name
,      avg(activity_running_time_in_msec)
,      MIN(activity_running_time_in_msec)
,      MAX(activity_running_time_in_msec)
FROM   bpm_activity_performance_v
GROUP BY process_name
,      activity_label
,      role_name
;

You could add the revision to see whether certain improvements in the process resulted in faster handling of activities by joining with the BPM_PROCESS_DEFINITION_V. It could also be usefull to join with the BPM_ACTIVITY_DEFINITION_V and discriminate on ACTIVITYTYPE.

Scrum guide mind map

Just found out I forgot to publish the Mind Map I used while studying for my scrum master certification. The mind map is based on the scrum guide. Download the scrum guide mind map pdf.

Scrum Team

  • Product owner: Responsable for maximizing the value of the product. She manages the Product Backlog.
  • Development Team: Does the work to deliver the releasable increment. The team is self-organizing and because of that needs to be cross-functional. It’s size shoud be somewhere beteen 3 and 9.
  • Scrum master: A servant leader who makes sure scrum is understood and enacted.

Scrum Events

  • Sprint: Time-box (month) to create the useable, potential releasable product ‘Increment’. Can be cancelled only by the Product Owner
  • Sprint Planning Meeting: Or actually a part focussing on the What and part on the How. On the what side focus on Forecast functionality based on the Product Backlog, Latest Increment, Projected capacity of the Development Team and Past performance of the Development Team. The How boils down to the so called Sprint Backlog.
  • Daily scrum: is performed by the Development team and answers 4 questions in 15 minutes: What has been accomplished? What will be done next? What obstacles are in the way? And syncs activities and plan for next 24 hours.
  • Sprint Review: Held to inspect the increment and adapt the Product Backlog. Time-boxed at 4 hours.
  • Sprint Retrospective: Where the Scrum Team inspect itself on people, relations, processes and tools. Besides that it plans for improvements (adapt). Time-boxed at 4 hours.

Scrum Artifacts

  • Product Backlog: with description, order/priority and estimate. Maintained by backlog grooming.
  • Sprint Backlog: Monitor sprint progress on the (total) work remaining. Set of Backlog items selected for the Sprint + a plan for delivering the product Increment and realizing the Sprint Goal.
  • Increment: The sum of all the Product Backlog items completed during a Sprint and all previous Sprints.

Definition of done

The definition of done is used to have a shared understanding of “done”. You can find a good definition of done example on this blog.

Selected software development trends

About 2 months ago InfoWorld published 11 programming trends to watch. I’d like to share three with you since they are close to home for me:

  1. No code is an island
  2. Bandwidth is no longer free
  3. Energy is no longer free, either

No code is an island

Having worked in integration project for almost a decade the idea that there is little code living on an island is not strange to me. However InfoWorld points out that besides that more and more software developer are creating products to enhance other products

Our code is living increasingly in ecosystems. Many PHP programmers, for instance, create plug-ins for WordPress, Drupal, Joomla, or some other framework. Their code is a module that works with other modules.

The same goes for development for mobile devices that rely increasingly on modules or apps created by others, whether they run on the device or in the cloud. This increases the demand for stable interfaces and contracts. Besides that the requirements for availability and scalability will weigh in heavier.

An urge for lean programming

Or create programs that deliver value in an efficient way. New releases of software programmers tend to demand always more resources (just a small example). The cost of keeping a computer plugged in has never been an issue. It never mattered how much energy your rack of servers sucked down because the colo just sent you a flat bill for each box.

The Cloud trend tends to make cost more transparent. Some of the clouds — like Google App Engine or Amazon S3 (example) — don’t bill by the rack or root password. They charge for database commits and queries. This adds a new perspective for software developers. We might need to start thinking about the cost of each subroutine in euros, not in lines of code, function points or milliseconds of execution time.

On the consumer side more and more ISPs adding bandwidth caps and metering. To a software developer this means that optimizing bandwidth consumption when designing apps is becoming imperative. Besides the cost issue this will also be needed because of the customer experience (loading speed etc).

Lean, agile and SOA reading list of 2011

Since this blog is also dedicated to sharing resources that are valueable to me I decided to share my reading list of 2011 with you.

Lean Integration: An Integration Factory Approach to Business Agility


A great best practices book on integration. The first part provides description of the business value of Lean. It introduces the core concepts. As a manager that doesn’t need all the details you could just read this part and you can get a good grasp of the ideas presented.

The second part translates the lean principles from the world of manufacturing to the world of systems integration. It has great case studies that shows the principles applied in a real world context.

Part three of the book provides a “how to” guide. This can be used as a reference and as such is a great desk-top reference manual. This book is great and a must read for all technology and business practitioners and innovators.

Web Service Contract Design and Versioning for SOA

Great reference (not a book that I read front to back) on Web Service Design from Thomas Erl and his co-authors. This book focuses exclusively on the contract part of the service. Due to the depth it is a extensive resource to use besides others. The book is filled with extensive examples on how to meet the goals of SOA properly using contract design.

Via the site of the publisher and on iTunes are additional service design podcasts by the authors of the book. Could be a great resource to start with.

The Back of the Napkin (Expanded Edition): Solving Problems and Selling Ideas with Pictures


This is a great book on problem solving, extremely useful and in a sense thought provoking. It structures problem-solving into a six by five visual codex. This makes sense; you can literally see the evolution of the thought processes and the development of the insights take shape through the pages. Fun read as well.