The milestone release model revisited

In my Percona live keynote “Driving MySQL Innovation” on Tuesday I talked about the quality problems we had with 5.0 and 5.1 and the switch to the new Milestone Release model. I explained how this new development process allowed us to deliver both 5.5 and 5.6, on time, including innovative features, and with very high quality.

The description of the Milestone Release model has been available here, for almost two years. So, this is not new but it has now been in operation for 3.5 years and it can be worth looking back and summarize.

Our first MySQL Server Milestone 5.4.3 was released on October 9th, 2009 and was then followed up by 5.5.2 on Feb. 26, 2010. Initially there was a time of trying out and fine tuning the model before it found its current form around 5.5 GA. The stabilized model has been followed consistently throughout the 5.6 development cycle.   Our latest milestone release (5.7.1) was released this week and was first 5.7 milestone release. In total we have now done 11 milestones in 5.5, 5.6 and 5.7 together. In short we refer to these releases as DMR’s (Development Milestone Releases).

Why did we create the milestone release model? The milestone release model came into existence because our previous models failed badly. The problems were many: First, we were not able to ship on time (not even close), and in some cases not at all (maybe a blog for another day, “The MySQL releases that never shipped”). Second, quality was bad or unknown. Third, developers were frustrated. All these issues were fixed by the process change.

So what is the essence of the Milestone Release model? The heart of the milestone release model can be explained by three words: “mysql-trunk is sacred”. The key is to keep soundness and quality in our main development branch at all time. Two secrets here: One is to have massive regression testing on trunk at all times and to immediately fix anything that is broken. Second, to do massive QA on all new features before they are allowed into trunk. New features are developed in independent feature trees while frequently pulling in changes from trunk to keep it in sync. When feature development is done it goes to QA and will stay in QA until all bugs found by QA are fixed and a sign-off is given by QA. At this point, *after* QA is completed, the feature will be merged to trunk.

To summarize our experiences: With the Milestone Release model our historical problems are gone: A problematic feature will be stopped before it comes to trunk and will thus not pollute trunk. We are able to ship trunk at any time since quality is maintained on a daily basis. Developers are no longer blocked by a bad trunk, their feature goes in when it is ready.

This process is working out great internally. The process allows our enginners to focus on their work. However, the biggest benefit is to our customers and users as they are getting predictability, timeliness and quality.

Celebrating 10 years @MySQL

In early Spring 2003 I called Erik Granström @MySQL, wanting to discuss if we could work with MySQL on a new storage engine.  He directed me to Mårten Mickos, the CEO of MySQL. After a brief call with Mårten, we were to take up the discussion later in the Spring.  Mårten was very busy, and did not have time to take the discussion further, he did not tell me why, but I learned it a few months later.

At the time I was heading up a database start-up.  We had struggled for some time with customers, as they all wanted a “standard interface” to access the data, and all we could offer was a proprietary C++ interface. The answer lay in SQL, and we had done some initial work on an ODBC driver, where the parser and query execution was all done in the api.  It was tedious work, and clearly not where our key differentiator was.  Reading in the Linux Journal that Winter about MySQL seemed to present a solution to our problem.  MySQL had all those upper layers of the database, and a storage engine interface – “just plug it in and we would be ready to go” – if it had been that simple :-)

Today you know the result of this integration as MySQL Cluster, which is being used in every corner of the world, powering mobile networks. Back in 2003 we used the tagline – “The Telecom Database” – in our marketing material. And looking back at my own career, it is one of the things I’m most proud of – We made it!

That Spring, Mårten landed the Series B funding for MySQL and a major deal with SAP. I guess he was excused for putting me off for a while :-)   The discussions reconviened, but it turned out to be a very different one – we got acquired.  This is a story in itself, meriting a separate chapter one day – meeting Monty and David for the first time – meeting Larry Stefonic – and batteling the Swedish union.

That’s how my journey with MySQL started. I’ve had a great first 10 years with MySQL, and I’m looking forward to many more years to come. I remain faithful to MySQL, and I believe in the path we are paving for MySQL here at @Oracle.

Thanks for reading, and stay tuned. Tomas