MySQL University: Developing MySQL on Solaris/OpenSolaris

We have a MySQL University session later today, featuring me and Trond Norbye (who works on Memcached development). Ostensibly we are talking about some of the tools and environment details of developing on Solaris and OpenSolaris, but with a MySQL focus. I’ve just updated the Forge page with the requisite information and a copy of the slides if you need them. Here’s the abstract:

Developing MySQL on Solaris requires you to install a suitable compiler and other tools, but you may be surprised to know that most of the material is there already, or easy to install if it’s not. But even more so, there is a huge wealth of information that you can get about your application while it’s running, both with and without using more traditional debugging methods. We’ll cover setting up a suitable environment, where to find the things you need, and and how to make use of the process monitoring tools and debugging environment to get the best out of development on Solaris generally, and MySQL specifically.

Please attend if you have any interest in developing on Solaris/OpenSolaris and MySQL. Trond an I will also be on-hand to answer any questions you might have.

Multiple VCS Updates and Cleanups

I spend a lot of time updating a variety of different repositories of different varieties and denominations, and I hate having to do that all by hand – I’d rather just go up into a top-level directory and say update-all and let a script work out what to do, no matter what different repos are there. I do it with a function defined within my bash profile/rc scripts, and it covers git, bzr, svn, bk, and cvs. The trick is to identify what type of directory we are updating. I do this, lazily, for each type individually, rather than for each directory, but I’ve found this method to be more reliable.

update-all () { for file in `ls -d */.svn 2>/dev/null`; do realdir=`echo $file|cut -d/ -f1`; echo Updating in $realdir; ( cd $realdir; svn update ); done; for file in `ls -d */.bzr 2>/dev/null`; do realdir=`echo $file|cut -d/ -f1`; echo Updating in $realdir; ( cd $realdir; bzr pull ); done; for file in `ls -d */.git 2>/dev/null`; do realdir=`echo $file|cut -d/ -f1`; echo Updating in $realdir; ( cd $realdir; git pull ); done; for file in `ls -d */CVS 2>/dev/null`; do realdir=`echo $file|cut -d/ -f1`; echo Updating in $realdir; ( cd $realdir; cvs up ); done; for file in `ls -d */BitKeeper 2>/dev/null`; do realdir=`echo $file|cut -d/ -f1`; echo Updating in $realdir; ( cd $realdir; bk pull ); done; unset realdir}

That’s it – a quick way to update any directory of repos.

Solaris 08/07 impressions

New versions of Solaris keep coming out apace, and with each new version we see improvements and enhancements, both from the core Solaris group and from the cooperation with the OpenSolaris teams. Solaris 10 8/07 came out last year, but it takes a while to go through and look at all of the new features, and in the time it’s taken me to fully understand all the new stuff, they’ve come out with another new version, but the functionality in the 8/07 release warrants some closer inspection. You can get the full details with the official documentation.First up, this is one of the the first releases where I feel comfortable using it pretty much out of the box without modification. Most of the tools and functionality are available with a standard install. You get Firefox 2.0 and Thunderbird 2.0 as standard, and combined with the improvements to the X Window System, we have an OS that can be used as a desktop without any additional installs. There are still some things I prefer to re-install (perl, for example), but the point is you no longer have to install many things. With Sun Studio being available, there’s little reason not to use Solaris 10 as a good replacement for Linux – in fact, one of the other elements I want to talk about might help there. Of course, its not all plain sailing. One of the problems I encountered is that I installed onto a machine with three identical drives – but when prompting you to partition disks, it shows you all of the disks without any kind of indication which one is which. You just get the size of the volume to choose from (a little difficult to choose when your machine has three identical drives). Luckily this was an empty machine that I could trash, but I wouldn’t want to identify the drive during installation just based on the partitions each device had. Actually, it turned out worse than this – after installation, Solaris had actually ended up on my second HD (the first attached to the second IDE controller), not my first. I had to swap the disks around to get it started properly. Really it’s a minor issue, but one to be wary of.One of the big improvements throughout are the changes and enhancements to the ZFS file system. One of my favourites is that you can now create a ZFS pool and then share it as an iSCSI device to other systems. This means that if I had a big server with loads of space, I could create a ZFS pool, get the benefits of ZFS on the server side, and share portions of the disk for storage over to other machines. Enabling iSCSI sharing is just a case of setting the shareiscsi

# zfs set shareiscsi=on mainpool/ishare

On a potential client, you add the server sharing the pool to the list of iSCSI targets, and then run format. With iSCSI, you are effectively sharing the physical device, so it just appears like a another disk, which you partition and organize as you want. I can see a real benefit here for providing users with a dedicated disk that they can organize how they like, while still giving you control the disk space and allocation at the server end. ZFS also has a huge range of other improvements. There’s no ZFS booting in this release (it’s in b84 of the regular OpenSolaris releases), but ZFS support and facilities just keeps going on and up. Using a single OS is often no longer a reasonable choice. Different companies release their products on different OS, and sometimes you just have to install another OS. Virtualization is one solution (and there’s plenty of news about that in later builds of Solaris), but in 8/07 we have another alternative, BrandZ. BrandZ builds on Solaris containers – logical divisions in the running environment that allow you to divide, allocate and isolate resources on a system into discrete, well, containers. Each container is effectively running another instance of Solaris (although it’s not as resource hungry as I make that sound). BrandZ extends that container functionality with an additional layer that provides binary compatibility for other operating systems, without going to the same level of complexity or resource requirements as the virtualization options. There are limits, of course, to that – you are not running a copy of the corresponding OS kernel, instead, an interface layer is providing the compatibility between the linux kernel and library calls and the corresponding interfaces on the Solaris side.You can’t run device drivers, and you also can’t run applications that require access to the graphics card; you have to run any X applications by using an X server in the global zone. That means you can’t run in Linux, but since you can attach to the main X service, it still means you can run Firefox and other applications running as Linux binaries but displaying on your main host (or indeed, anywhere with a running X server). For the most part, that means you can run Linux binaries in a BrandZ zone without resorting to full virtualization. It works fine- I was able to run a whole range of apps without any problems. The downside? Currently it’s x86 only. I’d love to be able to run SPARC Linux apps on my SPARC servers, for testing purposes only it would fill a hole I otherwise have to fill with another machine running another OS. BrandZ is a convenient way to run Linux applications without running a full Linux distro either on another machine or in a virtualization. This means it will appeal either to the user with a Linux binary legacy application, or the casual Linux application user without the complexities of a VM layer or dual booting. A good example here is something like Skype, available on Linux, but not Solaris. There are many examples of people using BrandZ on Solaris (see here for an example). All in all, there’s a lot to like in this release. ZFS is maturing and extending nicely, and I certainly want to play with the iSCSI feature a bit more to see where it can be used effectively (I’ve got a new server on it’s way soon to help with that test) .BrandZ, on the other hand, will solve the issue of running Linux applications to the point where I dont need VMware or Parallels for some jobs. I wont completely eliminate the need, but for one of my desktops it will make life much easier.

An introduction to Eclipse for Visual Studio users

I’m seeing more and more people moving to Eclipse as a development platform, even those Windows users who have traditionally used Visual Studio. As an Eclipse user for quite a while now I’m often asked how good it is, or how to use it. Of course, telling people to simply try it out isn’t enough. Many people just don’t get Eclipse and cannot understand or translate the skills and experience they already have to the Eclipse environment. That’s where An introduction to Eclipse for Visual Studio users can help. It’s a quick overview of the fundamentals of Eclipse from the perspective of a Visual Studio user. For a more in depth examination, there’s a tutorial Eclipse for Visual Studio developers, and another on migrating your applications from VS to Eclipse: Migrate Visual Studio C and C++ projects to Eclipse CDT.I can recommend any (or indeed all) of these.