MySQL 5.1 in OpenSolaris

If you’ve attended just one of my recent talks, either at the UC, LOSUG or MySQL University, you should know that MySQL 5.1.30 will be in the next official drop of OpenSolaris. In fact, you can find MySQL 5.1 in the current pre-release builds – I just download build 111 of the future 2009.06 release. Key things about the new MySQL 5.1 in OpenSolaris:

  1. Contains the set of DTRACE probes that also exists in MySQL 5.4 (see DTrace Documentation)
  2. Like the 5.0, we have SMF integration, so you can start, stop, monitor and change some of the core configuration through SMF
  3. Directory layout is similar to 5.0, with a version specific directory (/usr/mysql/5.1), and the two can coexist if you want to handle a migration from 5.0 to 5.1

To install MySQL 5.1, use the pkg tool to perform the installation. We’ve split the components into three packages:

  1. SUNWmysql51 contains the server and client binaries, and associated scripts.
  2. SUNWmysql51libs contains the client libraries, which you’ll need for all external (i.e. MySQL) tools, like DBD::mysql for Perl connectivity)
  3. SUNWmysql51test contains the MySQL test suite

To install:

$ pfexec pkg install SUNWmysql51

Once installed, you can start MySQL server through SMF:

$ pfexec svcadm enable mysql:version_51

You can set properties, like the data directory, by using svccfg:

$ svccfg svc:> select mysql:version_51 svc:/application/database/mysql:version_51> setprop mysql/data=/data0/mysql svc:/application/database/mysql:version_51> setprop mysql/enable_64bit=1

Any questions, please ask!

MySQL Docs Updates and Stats

We got an interesting question on the documentation list the other day, which basically asked if we provided a service that listed changes to specific pages in the manual. While I like the idea of such a service, the mechanics of making such a service work are very difficult. To start with, and to re-iterate something I have to explain again and again:

the MySQL documentation is rebuilt up to 10 times *every* day

We don’t have set schedules for when we release changes. We don’t bundle changes up and then produce a new reference manual on a set day of the month, or week. If I make a change to the documentation right now, there is every chance that you would see that change in the live docs at within 3 hours, and for the PDF format, possibly even within the hour. Yep, it happens that quickly. And that happens every day of the year – even at weekends and holidays. It’s also worth pointing out some of the statistics of the documentation to help explain why such a system might be impractical, but not impossible.Our current mysqldoc repo, in existence since we made the move to DocBook, has just hit revision 14955, which means over the last 3 years and 7 months, we’ve made on average 11.45 commits every single day. On some days, we make many more than that, and sometimes those changes can be minor (like a typo correction), and other times they will be a huge reorganization or rewrite.But there-in lies the other problem with any kind of monitoring of changes – our documentation is big and complex. Any ‘page watcher’ to notify you of changes would be limited by the fact that we split the reference manual into just over 2000 different HTML pages. And of course we do that for each manual (4.1, 5.0, 5.1, 5.1-maria, 5.4, and 6.0). For those keeping score, that’s about 12,000 different HTML pages for the reference manual alone. If you want to talk printed pages (and why not), the Letter-sized PDF manuals for the main documents in English (reference manuals, GUI docs, the MySQL version reference (15 docs) constitutes about 17,300 pages of content. We actually create 91 different documents from our English language repository, and 234 documents across all languages in up to 14 different output types (PDF, HTML, texinfo, txt, etc). So to return to the original question, would a page monitoring service be possible? Probably. Would it be useful? I’m in two minds about this. I can see the value of having a way to see changes, but if what you want is to be able to monitor changes in behavior, then most of the time this is documented in the changelog, and we have a number of different outputs available for viewing the changelog information in more useful ways. Other, more minor changes, and in some case major rewrites wouldn’t feature in the changelog (because they don’t relate to a change in the product we are documenting). Whether you would find such changes useful is up to you. A rewrite normally means we’ve decided to change the content to make it clearer, or we’ve re-organized the information to make it flow better. Again, it doesn’t necessarily constitute a change to the product that wouldn’t otherwise be in the changelog.Would it be practical? Probably not – the sheer size of our documentation would mean that just providing the data on the changes would probably double the number of pages and information, and it wouldn’t always be clear whether the change were just a typo correction or a major rewrite. And tracking the changes if we change a page ID (which we do, from time to time), would it difficult to correctly identify changes across different pages that really had the same content. Of course, I’m open to suggestion here, and we are doing things to further improve and enhance the content, and maybe this is something we will consider. But I’m open to input on whether this is needed.

UNIX network analysis

I have a new tutorial on analyzing networks, in terms of understanding your basic network configuration, the other machines and devices on the network, and the general topology. From the intro:

When accessing a new UNIX system, or even understanding an existing one, a key part of the puzzle to how the system operates is the network configuration. There are many aspects of the network that you need to know and understand to correctly identify problems and prevent future problems. By using some basic tools and commands you can determine a lot about the configuration of a single system, and through this basic understanding, a good idea of the configuration of the rest of the network. With some additional tools, you can expand that knowledge to cover more systems and services within your network.In this tutorial you will use some basic tools within the UNIX environment that can disclose information about the configuration of your system. By understanding these tools and the information they output, you will be able to gain a greater understanding of your system network configuration and how it works. You will also examine tools and solutions that can look at the wider network and gain more detailed information about your network, its potential security issues, and key points of information that will help you identify and diagnose problems when they do occur.

Read UNIX network analysis

Solutions for tracing UNIX applications

Tracing applications are something of a passion for me, especially with the introduction of DTrace in Solaris and Mac OS X. To support that, I have a new tutorial about the different methods available for tracing Unix applications. I tried to concentrate on tools and techniques that don’t require access to the source, like using truss and DTrace. From the intro:

Most developers and systems administrators know what should happen in their operating system and with their applications, but sadly, this isn’t always the case. There are times when an application has failed, or is not behaving as you expect, and you need to find out more information. By using your existing knowledge of how your application should work and some basic UNIX skills, you can trace the application to find out what is causing the problem. This tutorial will teach you the basic techniques of using tracing tools to find out what your application is doing behind the scenes.First, the tutorial looks at the distinction between debugging and tracing, and how the two solutions differ. Then it examines some specific examples of where tracing can be used to solve problems in your application. DTrace provides elements of both system tracing and debugging, and also provides you with the ability to time and benchmark applications. Finally, the tutorial shows how to trace the information being exchanged between network computers to help find problems in network applications.

Read Solutions for tracing UNIX applications

Synchronizing UNIX files

I have a new article on different ways in which you can synchronize your Unix files. From the intro:

There are many tools available that allow you to synchronize files across UNIX directories, but doing it effectively, and securely, takes a little bit more effort. This article looks at solutions for synchronizing files across UNIX filesystems and different computer systems securely, and at solutions that allow you to synchronize encrypted versions of your files for the purposes of backup.File synchronization is the process of adding, changing, or deleting a file in one location, and having the same file added to, changed, or deleted at another location. This article covers three utilties, cp, tar, and rsync, that can aid with synchronization of UNIX files. While cp and tar commands have limited synchronization abilities, rsync provides you with the full range of options; however, all three have their place.

Of particular interest is a script that provides an encrypted wrapper around rsync, which I use to provide a nice secure rsync-able backup. Read: Synchronizing UNIX files

Speaking at CommunityOne West

Sorry for the (relatively) short notice, but I will be talking at Sun’s CommunityOne conference in San Francisco on June 1st.I’ll be talking about, and demonstrating, the DTrace probes we have put into MySQL in a joint presentation with Robert Lor who will be doing the same for Postgres.


CommunityOne West Badge

Our presentation is on the Monday afternoon. Check out the CommunityOne West Conference Site for more details and registration.

Solaris 10 Boot Failure

OK, so this has now bitten me twice on a new install. Basically, put Solaris 10u5 on certain machines, and it will work fine until you edit the /etc/vfstab and forget to add a terminating newline to one of your entries. Upon reboot you will get something like this: Error: svc:/system/filesystem/root:default failed to mount /boot (see 'svcs -x' for details)[ system/filesystem/root:default failed fatally (see 'svcs -x' for details) ]Requesting System Maintenance ModeConsole login service(s) cannot runAt the top of the output from svcs, you’ll see:svc:/system/filesystem/root:default (root file system mount)Reason: Start method exited with $SMF_EXIT_ERR_FATALsee: /etc/svc/volatile/system-filesystem-root:default.logImpact: 44 dependent services are not running. (use -v for list.)The problem is that missing newline, which means the mount table is never parsed correctly. To fix, enter your root password to get into admin mode. You’ll need to first remount the root fs as read/write: # mount -orw,remountAnd then add that offending missing line end: # echo >>/etc/vfstabBe careful with that second line; miss a > symbol and you’ll wipe out your vfstab altogether. Now reboot, and things should be back to normal.