April MySQL Engineering News

On a regular basis I plan to summarize the latest news from MySQL Engineering. I hope you find it useful.

April highlights were the DMR’s coming out for both MySQL Cluster 7.3.2 and MySQL Server 5.7.1. For those that have been missing the launchpad versions of those, I apologize for the delay, but they should be there now if you want to dig into the changeset details. And to repeat what I’ve said in the past, there should not be a delay between releases on launchpad and src tar balls, so please keep bugging me when you see glitches there. Personally I also very much enjoyed the April Percona conference where I met a lot of old good friends and gave a keynote. You can see it here or just read the presentation. Here follows a summary of the engineering blogs from last month.

MySQL Support
The MySQL support team has some excellent blogs well worth reading.  The latest month includes for example “Migrating to InnoDB Full text search…”, “MySQL Utilities…”, and plenty of recent 5.6 info.

MySQL Performance Schema (PS)
This is a tremendously powerful tool, but it can be overwhelming to familiarize yourself with. Learn more about configuring performance schema from the author Marc Alff. Feel free to give him feedback directly on his blog, the good as well as constructive feedback. I’d also like to draw your attention to an older blog by Mark Leith, ps_helper. We are currently considering how we could include something like that in the product to make is easier to use PS. Ideas and input are greatfully received.

MySQL InnoDB
InnoDB continues to be the storage engine we focus on together with NDB for MySQL Cluster. You can learn more about the Data Organization in InnoDB, and the new 5.6 feature InnoDB transportable tablespaces and how to use them. Also have a look at Sunny’s presentation about 5.6 InnoDB features and scalability at Percona Live in April

MySQL Cluster and NoSQL
I mentioned the new DMR 7.3.1 for MySQL Cluster. Our efforts to improve ease of use of Cluster continues, please learn more about, and try out, the autoinstaller. MySQL Cluster was there as a “NoSQL” database long before the term was coined, and was just awarded the “storage engine of the year” the other week. This DMR also contains support for Foreign Keys which I know some of you have been waiting for. I’ll encourage the cluster engineers to write an in depth blog about the complexities in supporting foreign keys in a distributed databases. Meanwhile please get an overview here.

MySQL Performance
Dimitri is continuing his great work on benchmarking MySQL 5.6, comparing it with different 5.5 versions, including MariaDB and Percona servers. His blogs are excellent reads for anyone interested in MySQL performance.

MySQL Testing
In my keynote at Percona Live I talked about the big investment we have made in QA for MySQL. Please take a moment to learn about MySQL Server QA.

MySQL Replication and Big Data
MySQL 5.6 contains many new replication features, learn about the lesser known but equally important new replication features. Luis gave an excellent presentation at Percona Live on MySQL Replication, where you can learn more about performance enhancing features like multi-threaded slave and binary log group commit, as well as global transaction identifiers that can help you to do failover much better.

In my keynote at Percona Live I talked about how MySQL plays in important part in the Big Data lifecycle, and the new Hadoop applier for MySQL. Here is some more reading from Shubhangi, Hadoop applier part 1 and part 2.

MySQL Optimizer
MySQL 5.6 contained a lot of Optimizer enhancements. Here is a blog on subqueries from Øystein, a senior member of the Optimizer team, also aggregated to the MySQL Optimizer blog if you are interested in older posts.

MySQL on Windows
Our focused effort to on MySQL on Windows continues. New this month is the beta of the Connector/Net 6.7.2 which includes support for connecting to the MySQL memcached interface as well as load balancing. If you have not yet tried out MySQL for Excel I encourage you to try the newest version. You can follow the development of MySQL on Windows blog if you are interested.

MySQL General
Please have a look at Mikael’s insights about MySQL Engineering. And read about the fixing of a long standing trigger bug which is also aggregated on the MySQL Runtime blog.

MySQL Cluster gets a Community Award

It was with pride I entered the stage at the Percona MySQL Conference yesterday to receive the “Storage Engine of the Year 2013 award” for ndb (the MySQL Cluster storage engine).  Thank you very much, on behalf of myself and the MySQL Cluster engineering team. The team has done a great work over the past 10 years to harden the product, and add new features. MySQL Cluster milestone 7.3.2 was just released. It will be my pleasure to present the “trophy” some members of the team when I get back to Stockholm. Please see a previous post for how the MySQL Cluster journey began for myself and the ndb team 10 years ago.

The milestone release model revisited

In my Percona live keynote “Driving MySQL Innovation” on Tuesday I talked about the quality problems we had with 5.0 and 5.1 and the switch to the new Milestone Release model. I explained how this new development process allowed us to deliver both 5.5 and 5.6, on time, including innovative features, and with very high quality.

The description of the Milestone Release model has been available here, for almost two years. So, this is not new but it has now been in operation for 3.5 years and it can be worth looking back and summarize.

Our first MySQL Server Milestone 5.4.3 was released on October 9th, 2009 and was then followed up by 5.5.2 on Feb. 26, 2010. Initially there was a time of trying out and fine tuning the model before it found its current form around 5.5 GA. The stabilized model has been followed consistently throughout the 5.6 development cycle.   Our latest milestone release (5.7.1) was released this week and was first 5.7 milestone release. In total we have now done 11 milestones in 5.5, 5.6 and 5.7 together. In short we refer to these releases as DMR’s (Development Milestone Releases).

Why did we create the milestone release model? The milestone release model came into existence because our previous models failed badly. The problems were many: First, we were not able to ship on time (not even close), and in some cases not at all (maybe a blog for another day, “The MySQL releases that never shipped”). Second, quality was bad or unknown. Third, developers were frustrated. All these issues were fixed by the process change.

So what is the essence of the Milestone Release model? The heart of the milestone release model can be explained by three words: “mysql-trunk is sacred”. The key is to keep soundness and quality in our main development branch at all time. Two secrets here: One is to have massive regression testing on trunk at all times and to immediately fix anything that is broken. Second, to do massive QA on all new features before they are allowed into trunk. New features are developed in independent feature trees while frequently pulling in changes from trunk to keep it in sync. When feature development is done it goes to QA and will stay in QA until all bugs found by QA are fixed and a sign-off is given by QA. At this point, *after* QA is completed, the feature will be merged to trunk.

To summarize our experiences: With the Milestone Release model our historical problems are gone: A problematic feature will be stopped before it comes to trunk and will thus not pollute trunk. We are able to ship trunk at any time since quality is maintained on a daily basis. Developers are no longer blocked by a bad trunk, their feature goes in when it is ready.

This process is working out great internally. The process allows our enginners to focus on their work. However, the biggest benefit is to our customers and users as they are getting predictability, timeliness and quality.

Celebrating 10 years @MySQL

In early Spring 2003 I called Erik Granström @MySQL, wanting to discuss if we could work with MySQL on a new storage engine.  He directed me to Mårten Mickos, the CEO of MySQL. After a brief call with Mårten, we were to take up the discussion later in the Spring.  Mårten was very busy, and did not have time to take the discussion further, he did not tell me why, but I learned it a few months later.

At the time I was heading up a database start-up.  We had struggled for some time with customers, as they all wanted a “standard interface” to access the data, and all we could offer was a proprietary C++ interface. The answer lay in SQL, and we had done some initial work on an ODBC driver, where the parser and query execution was all done in the api.  It was tedious work, and clearly not where our key differentiator was.  Reading in the Linux Journal that Winter about MySQL seemed to present a solution to our problem.  MySQL had all those upper layers of the database, and a storage engine interface – “just plug it in and we would be ready to go” – if it had been that simple :-)

Today you know the result of this integration as MySQL Cluster, which is being used in every corner of the world, powering mobile networks. Back in 2003 we used the tagline – “The Telecom Database” – in our marketing material. And looking back at my own career, it is one of the things I’m most proud of – We made it!

That Spring, Mårten landed the Series B funding for MySQL and a major deal with SAP. I guess he was excused for putting me off for a while :-)   The discussions reconviened, but it turned out to be a very different one – we got acquired.  This is a story in itself, meriting a separate chapter one day – meeting Monty and David for the first time – meeting Larry Stefonic – and batteling the Swedish union.

That’s how my journey with MySQL started. I’ve had a great first 10 years with MySQL, and I’m looking forward to many more years to come. I remain faithful to MySQL, and I believe in the path we are paving for MySQL here at @Oracle.

Thanks for reading, and stay tuned. Tomas