Mon 17 Oct 2005
Last month, we upgraded iOcean to Mac OS 10.4.2. Since then, a series of ‘ripples’ from the update have occurred that show how much impact an environment change has beyond the initial three hours of downtime.
1) After the upgrade, we needed to upgrade to a newer version of the server administration tools from Apple (the 10.3 tools can’t be used to administer a 10.4 server).
2) In order to run to 10.4 administration tools, the OS on the computers from which iOcean is administered had to be upgraded to 10.4 as well.
3) One of the workstations from which iOcean is administered (my workstation, in fact) did not have a DVD drive, and thus OS 10.4 had to be installed on it over the network
4) We did not have a NetBoot/NetInstall server, so the service need to be set up on one of our servers, and the NetInstall images created, a process that took several hours of reading, installation, and troubleshooting.
5) The workstation I was using turned out to be incompatible with NetBoot 2.0. Luckily, we had another, newer computer that was not in use, and became (after some hardware swapping) my new workstation.
In the end, the iOcean upgrade - which was fairly minor, as it didn’t involve a major version change to Apache, PHP, MySQL, or any other services on iOcean - resulted in a great deal of additional work. In this case, the additional work - the 10.4 upgrades, the NetInstall setup, and even the hardware swapping - are all things that were slated to happen soon anyway, so the effort was not really wasted.
However, it is also the case that this additional work could have been planned out ahead of time. For example, although I knew I would have to upgrade my workstation to 10.4, I didn’t think to check that we had compatible media until I sat down with the disk in my hand. Likewise with the NetBoot 2.0 incompatibility - it was discovered during the NetBoot setup process, not beforehand. Knowing that this incompatibility existed would have saved me a few hours in the long run. Having to deal with these issues as the after-effects of the iOcean upgrade, rather than projects in their own right, made them more pressing and meant there was less time for preventative research.
Of course, in hindsight it’s easy to see the problems one should have known about. In practice, there will often be issues that are overlooked, and as such there will often be ripples to deal with.
One Response to “Upgrades and the Ripple Effect”
Leave a Reply
You must be logged in to post a comment.


October 17th, 2005 at 3:54 pm
Though the title says it all, this captures nicely what we were after - the never-ending story of small nudges and unanticipated adjustments needed to not just maintain but rather restore alignment. Have to think about whether knowing this - by taking it from the informal everyday experience to a more formal documented recognition - is there a way to factor it into our planning better as we scale-up infrastructures so that we’re building not interupt-happening, patched up configurations but rather robust, sustainable environments.