MySQL University: Developing MySQL on Solaris/OpenSolaris

We have a MySQL University session later today, featuring me and Trond Norbye (who works on Memcached development). Ostensibly we are talking about some of the tools and environment details of developing on Solaris and OpenSolaris, but with a MySQL focus. I’ve just updated the Forge page with the requisite information and a copy of the slides if you need them. Here’s the abstract:

Developing MySQL on Solaris requires you to install a suitable compiler and other tools, but you may be surprised to know that most of the material is there already, or easy to install if it’s not. But even more so, there is a huge wealth of information that you can get about your application while it’s running, both with and without using more traditional debugging methods. We’ll cover setting up a suitable environment, where to find the things you need, and and how to make use of the process monitoring tools and debugging environment to get the best out of development on Solaris generally, and MySQL specifically.

Please attend if you have any interest in developing on Solaris/OpenSolaris and MySQL. Trond an I will also be on-hand to answer any questions you might have.

Problems with the CHM Refman builds

As some of you have noticed (and reported) we are having trouble with the CHM builds we do of the various MySQL Reference Manuals. This is something that we are looking into, but the reasons and issues behind it are complex. One of the basic reasons if the sheer complexity and size of the documents are now building – there are thousands of files in a standard HTML build, constituting more than 2,500 pages of material. Unfortunately, the final part of the build process, using Microsoft’s own HTML Help Compiler is something we have no control over, and Microsoft no longer support the product. Sometimes it produces a perfectly fine CHM from the source material. Sometimes, using the exact same material, it produces a build that is corrupt in some way. The CHM building process is automatic – I improved it a couple of years ago to be automated as much as possible using a Windows Powershell script that performs nearly all of the work and verifies the build process. But Microsoft’s tool will report a successful build even if the build actually produces a corrupt CHM file. Identifying a corrupt build is difficult; some of the CHM builds open fine on Windows but not in Linux CHM readers, some the other way round, some that don’t work in anything but which reported a completely fine build. We are looking at different ways of addressing the issues, including creating simplified output (removing some of the more esoteric stuff), and releasing CHMs less often that have been manually checked. I’ll let you know the progress, but until then, we’re going to be highlighting the CHM build issue on the download page.

My Sessions at UC2009

I’m speaking at the User Conference this year, with a half-day tutorial and three further sessions. The running theme is performance, both in terms of the performance of your queries, and in terms of scaling up. Scale Up, Scale Out, and High Availability: Solutions and CombinationsThis is the big tutorial. It’s difficult to resolve what I’ll be talking about into a few sentences, but think about all of the different technologies available here – replication, partitions, sharding, DRBD, memcached – I’ll be talking about all of them, and more importantly combinations of the different solutions and where the potential performance gains and pitfalls are. I’ll also be using the opportunity to demonstrate some of the more obscure combinations that you can use to provide the environment you need. How I used Query Analysis to Speed Up my ApplicationsFor query analysis, I’ll start with some of the basic methods available to us for performance monitoring, including EXPLAIN and DTrace, before I look at the query analysis provided by MySQL Enterprise Monitor. As an advisor to the group I’ve been looking at it for a while and used it on my own sites to identify a range of different query problems. Improving performance by running MySQL multiple timesIt isn’t talked about much, but there are times when running a single instance of MySQL doesn’t get you either the performance or environment that you need to support your applications. In this presentation I’m going to look at some of the benefits, from simply running multiple instances, to using solutions like VMware, Xen, LDOMs, BSD Jails, and Solaris Containers. Using MySQL with the Dojo ToolkitThe final presentation is something a little more fun. The Dojo Toolkit is a JavaScript kit for developing AJAX applications. There are some really fun things you can do with Dojo, but getting the best combination and cool and efficient with MySQL is an art. We’ll look at two quick examples; the first is a browsable interface to large quantities of data. The other is dynamic graphing using MySQL as the backend. If, within the bounds of any of these presentations there is something you would like covered, please let me know.

Diarizing – what goes around, comes around

OK, so I’ve just written about my love of Moleskines and it occurs to me that today’s bloggers (myself included) are just continuing a long tradition for some people to share their experiences. I write my memories in my Moleskines, not on a day to day basis, but whenever anything significant is happening in my life, or when I’m traveling about the world for business or pleasure, and for me that’s a vital part of me recording my memories, despite my otherwise excellent recollection skills. Hundreds of years ago Samuel Pepys and many others did the same thing – diarized what was going on in their lives and the world around them, and it became part of the content that records our world history. I know not every blog is that interesting (my own including), but as the title says, what goes around, comes around – we are just carrying on the tradition of those great diarists that came before us.

DTrace in MySQL: Documentation and a MySQL University Session

DTrace has been something that I’ve been trying to get into the MySQL server for more than a year now.After a combination of my own patches and working with Mikael Ronstrom and Alexey Kopytov we finally have a suite of probes in MySQL 6.0.8. Better still, after a short hiatus while I was busy working on a million-and-one other things, the documentation for those probes is now available: Tracing mysqld with DTrace. The documentation is comparatively light and deep all at the same time. It’s lightweight from the perspective that I’ve added very little detail on the mechanics of DTrace itself, since there is no need to replicate the excellent guides that Sun already provide on the topic. At the same time, I’ve tried to provide at least one (and sometimes two) D script examples for each of the groups of probes in the 6.0.8 release. So what next for MySQL DTrace probes? Well, the next version of the probes has already been worked on. I’ve been testing them for a month or so, and due to a problem with the probes on SPARC I couldn’t approve the patch, but I managed to resolve that last week. The new patch extends the probes to enable a more detailed look at certain operations, and it enables us to expand the probes to be placed anywhere within the server, including individual engine-level row operations. If you want a demonstration of DTrace in MySQL, some of the things you can monitor without using the user probes we’ve added, and those new probes I just mentioned, then you will want to attend the MySQL University session this Thursday (12th Feb), at 14:00UTC where I’ll be doing all of the above. As a side note, because I know there are people interested, last week I also finished the patch for these probes to go into the MySQL 5.1.30 version that we will be putting into OpenSolaris. Sunanda is working on getting that release out there as I type this.