Category Archives: Open Source

Presenting at Codemotion next week on web scale patterns

Peter Paul van de Beek CodemotionNext week I’ll be presenting in Amsterdam at Codemotion on web scale patterns in bol.com back office services. Codemotion is the biggest tech conference in Italy and one of the most important in Europe, with a network of more than 30k developers.

The title of my presentation is: “Applying “web scale” patterns in the bol.com back office”. In the session, we show you how we use “web scale” patterns to achieve scalability and flexibility in our back office software. We will guide you through how we apply patterns like CQRS, event sourcing, polyglot persistence and micro services to solve puzzles in our back office services. Interesting is that in our experience we don’t need these just to solve a technical problem. It helps us to solve some of our business problems!

See you on May 16 and 17 in Amsterdam. There is a short interview with me on Codemotion world.

ScaleScale on the stack behind Netflix scaling

Over at ScaleScale, a blog about all the good stuff when it comes to scaling, an interesting post was published on the stack behind Netflix scaling. Since Netflix is quite public about how they operate, the post put was together with stuff from around the internet.

Stack Behind Netflix Scaling Like Spotify Netflix is kind of famous for creating and scaling their culture. This gives some important context to the culture to understand how they scale their software stack and why it works. If you are interested in scaleable platforms and full stack development check it out.

Continuous Delivery at bol.com

Last month two of our software engineers Mihaela Tunaru and Mary Gouseti were invited to give a presentation of how continuous delivery is done at bol.com. The presentation gives a good insight in the state of continuous delivery at bol.com from a software engineering perspective.

In case you want to know more from the operations perspective check Mayfly on GitHub and the presentation below. Maarten Dirkse gave a talk Docker your user stories using Mayfly.

Mayfly is a development platform built by bol.com. Mayfly speeds up your service development by wrapping your scrum user story code in containers, testing it in an isolated, production-like environment and automatically enforcing your Definition of Done.

Lean, agile and Software development reading list of 2013

In this blogpost I’ll share a list of books I read during the first six months of 2013.

Hadoop – The definitive guide

Hadoop the definitive guideThis book proved very useful to get an introduction and solid background in Hadoop. I was reading it a little before starting an enhancement of MapReduce code. This made it possible to better understand the production code and how to make the changes.

Hadoop The Definitive Guide (amazon) is recommended for anyone interested in Hadoop stuff.

Essential scrum

Essential ScrumWanted to read Essential Scrum to renew and deepen my theoretical knowledge of Scrum. This is a great read for that purpose!

I like the visuals that are used and set it apart from other books on the subject. Besides that I liked the MindMap-like figures that support the stucture in the chapters.

The scope goes beyond the core of Scrum and does that well. It also touches on subjects like Multilevel and Portfolio planning, The role of managers in Scrum context, and Product Planning.

This is a great follow up read for anyone with basic Scrum training or certification. It doesn’t just offer the big picture but both details and examples on how to become more agile. It will help you deal with the complexities of implementing and refining Scrum.

Thinking Fast and Slow

Thinking Fast And SlowThe aim of Daniel Kahneman the author of Thinking Fast and Slow is to enrich the vocabulary of people talking at a watercooler, where opinions and gossip are exchanged. He wrote this book to influence the way they talk about judgements and choices of others. He has succeeded. As Economist has put it: Kahneman shows that we are not the paragons of reason we assume ourselves to be. When you realise this it put you and the world around you in a different perspective.

Mr. Kahneman is a person that understands like no other on the planet how and why we make the choices we make. He knows how to share his insights! This is a great read for any curious mind, escpecially those with an interest in how and why we make choices.

This book will change the way you think.

There is an interesting talk on Thinking Fast and Slow by Mr Kahneman at the The Long Now.

Seven Databases in Seven Weeks: A Guide to Modern Databases and the NoSQL Movement

Seven databases in seven weeksThe book Seven databases in seven weeks will take you on a tour visiting some of the hottest open source database today. This is typical software development reading.

It has a progressive style of offering insigts to databases and their capabilities. The open source databases covered are PostgreSQL, Riak, Apache HBase, MongoDB, Apache CouchDB, Neo4J, and Redis. These were chosen to span five database styles or genres: Relational, Key-Value, Columnar, Document and Graph.

This book is recommended for anyone looking for a solid introduction fo databases besides the traditional RDBMS. It will provide the knowledge you need to choose one database to suit your needs.

Selected software development trends

About 2 months ago InfoWorld published 11 programming trends to watch. I’d like to share three with you since they are close to home for me:

  1. No code is an island
  2. Bandwidth is no longer free
  3. Energy is no longer free, either

No code is an island

Having worked in integration project for almost a decade the idea that there is little code living on an island is not strange to me. However InfoWorld points out that besides that more and more software developer are creating products to enhance other products

Our code is living increasingly in ecosystems. Many PHP programmers, for instance, create plug-ins for WordPress, Drupal, Joomla, or some other framework. Their code is a module that works with other modules.

The same goes for development for mobile devices that rely increasingly on modules or apps created by others, whether they run on the device or in the cloud. This increases the demand for stable interfaces and contracts. Besides that the requirements for availability and scalability will weigh in heavier.

An urge for lean programming

Or create programs that deliver value in an efficient way. New releases of software programmers tend to demand always more resources (just a small example). The cost of keeping a computer plugged in has never been an issue. It never mattered how much energy your rack of servers sucked down because the colo just sent you a flat bill for each box.

The Cloud trend tends to make cost more transparent. Some of the clouds — like Google App Engine or Amazon S3 (example) — don’t bill by the rack or root password. They charge for database commits and queries. This adds a new perspective for software developers. We might need to start thinking about the cost of each subroutine in euros, not in lines of code, function points or milliseconds of execution time.

On the consumer side more and more ISPs adding bandwidth caps and metering. To a software developer this means that optimizing bandwidth consumption when designing apps is becoming imperative. Besides the cost issue this will also be needed because of the customer experience (loading speed etc).

SCA Diagrammer

SCAWhile doing some research on the web I came across a Service Component Architecture Diagram Generator. The script generates diagrams for applications based on the Service Component Architecture (SCA) model. This can be a great aid to understand the relations between components in large BPEL or SOA applications.

Download the tool here. The readme is over here. The script runs on Linux/Unix and has primarily been tested with Oracle Fusion Middleware 11g applications but should work with other BPEL or SOA frameworks.

Oracle releases NoSQL Database, Enterprise Edition 1.1.100

While the NoSQL Database Community Edition still waits for final license approval, Oracle has released it’s NoSQL Database, Enterprise Edition 1.1.100. The download is available on OTN.

The Oracle NoSQL Database is a distributed key-value database. It is designed to provide highly reliable, scalable and available data storage across a configurable set of systems that function as storage nodes.

The documentation page is still rather empty. However Technical Overview and a Data Sheet are already available.

Oracle NoSQL is a key offering in Oracle’s portfolio of Big Data products. The product is based on the Java version of Berkeley DB, an open source database used in embedded systems.

Update

VirtualBox Solved VBOX_E_FILE_ERROR 0x80BB0004

Recently I wanted to share a VirtualBox Appliance with a colleague. So I tried to export the created appliance. Unfortunate this failed with a:

VBoxManage.exe: error: Code VBOX_E_FILE_ERROR (0x80BB0004) – File not accessible or erroneous file contents (extended info not available)

So I thought why not just copy it… Oh wait, first compact it to save some space and time using VBoxManage modifyvdi some.vdi compact:

>VBoxManage modifyvdi Windows7.vdi compact
0%...10%...20%...30%...40%...50%...60%...70%...80%...
Progress state: VBOX_E_FILE_ERROR
VBoxManage.exe: error: Code VBOX_E_FILE_ERROR (0x80BB0004) - File not accessible or erroneous file contents (extended info not available)

When I tried to copy or zip the file there were errors accessing it. Somehow the VirtualBox Appliance was still running fine. The files that make up the appliance weren’t locked.

Use CHKDSK

My laptop (the host of the appliance) is is running Windows7 and I decide to run chkdsk and found:

CHKDSK controleert de bestandsgegevens (stap 4 van 5)...
Leesfout met status 0xc0000185 op offset 0x17410c9000 voor 0x10000 bytes.
Leesfout met status 0xc0000185 op offset 0x17410d6000 voor 0x1000 bytes.
Er zijn beschadigde clusters in bestand 37907 met naam \Users\ppvdbeek\VIRTUA~2\Windows7\Windows7.vdi vervangen.

This is Dutch stating that there were bad sectors in the vdi file that had to be corrected. After that, I could run the appliance, compact the disk and export the appliance successful.

On Linux you could use fsck (for “file system check”) in a similar case.

Installing Ubuntu 11.04 in VirtualBox

This post will show you how to install Ubuntu 11.04 in VirtualBox (Recently a post on installing Ubuntu 12.04 in VirtualBox became available and even more recent installing Ubuntu 13.04 in VirtualBox). In later posts this will be used to run: JRockit 64 bit, WebLogic Server (WLS) and Oracle Service Bus (OSB).

Prepare Ubuntu install in VirtualBox

A short recipe for preparing the Virtual Machine to install Ubuntu on it. The blog post Installing Ubuntu inside Windows using VirtualBox is more extensive on this subject…

  • Create a new Virtual Machine using the VirtualBox Manager
  • Click Next in the Welcome screen
  • Choose a Name for the Virtual Machine and select the Operating System (OS) before you click Next
  • Set the amount of memory you want to allocate to the Virtual Machine
  • When asked for the Virtual Hard Disk, check the Boot Hard Disk box and Create a new hard disk. Then the Create New Virtual Disk Wizard will start guiding you through the process
  • I choose a Fixed-size storage when asked for the Hard Disk Storage Type
  • Choose the Size (16 GB) and Location (left it default) for the Virtual Hard Disk
  • Check the summary
  • Watch the progress while the .vdi file is created
  • Click Finish to acknowledge the creation of the new Virtual Machine
  • The newly created Virtual Machine will show up in the VirtualBox Manager
  • In order to install Ubuntu 11.04 on the Machine go to the settings to alter the storage settings
  • Use the icon besides the IDE Controller to add the disk image – ISO file – you downloaded to boot and install Ubuntu as media in a drive
  • Check the boot order and make sure that the Machine is booted from the CD/DVD device first
  • Acknowledge
  • Start the Virtual Machine and the “regular” installation of Ubuntu will start

The screenshot that come with the short instructions above:

Ubuntu install

  • Boot the installation ISO
  • Choose Install Ubuntu and your language
  • Preparing to install Ubuntu – You don’t need the additions here to later install WebLogic Server and Oracle Service Bus
  • Allocate drive space – If you created a new Virtual Disk before you can safely erase the empty disk here
  • Erase disk and install Ubuntu – Acknowledge
  • Choose location and time zone
  • Keyboard layout
  • Who are you – Fill in the username, computer name and password
  • Welcome screen
  • Installation is complete – click the button to restart
  • Login as the user you just created

Watch the screenshots that come with the steps above:

Installing VirtualBox Guest Additions

  • From the VirtualBox Manager menu choose the Devices - Install Guest Additions options
  • Choose the application to start
  • Click Run
  • Authenticate with your password
  • You need to restart Ubuntu to make it work

Watch the screenshots that come with the steps above:

Stop websites from tracking you

An earlier post on how cookies are used to track you, explained how tracking cookies work. This post will show you how to stop websites from tracking you using Firefox 4. This latest release has a Do-not-track feature that lets you tell websites you don’t want your browsing behavior tracked.

By turning on the Do-not-track feature, Firefox tells websites you visit that you don’t want your browsing behavior tracked. Please note that honoring this setting is voluntary. To put it differently websites are not required to respect it.

Turning on the Do-not-track feature in Firefox 4

  • Click the Firefox button at the top of your browser window and click options.
  • Make sure you’re on the Advanced panel.
  • Select the General tab.
  • In the browsing section check the Tell websites I do not want to be tracked.
  • Click OK to leave
Do-not-track feature

Check the Tell websites I do not want to be tracked option