Many of my servers are configured with a software mirrored boot volume configured as per the FreeBSD handbook. Unfortunately the process outlined on those pages results in a mirror configuration that sometimes works in FreeBSD 9.0 and sometimes does not. Here is how I handle the situation where it does not. Firstly, I assume you know how to run a source upgrade, and you already have mirroring configured. The instructions are pretty clear in the handbook and the UPDATING file in the source tree. The trouble will become apparent on the initial reboot into the new kernel. The tell-tale sign is that you get the following error: GEOM_PART: integrity check failed (mirror/gm0, MBR) and you end up with the mountroot> prompt. If you do not get this problem, then your gmirror will work fine with FreeBSD 9.0, and you don't need to do anything otherwise special to survive this transition.If you do get this error, the fix is to break the mirror, and re-create it using the GPT partitions which FreeBSD 9 prefers. My examples assume two SATA drives which in FreeBSD 8 and earlier show up as /dev/ad4 and /dev/ad6 .Breaking the Mirror
gmirror remove gm0 ad6 gmirror remove gm0 ad4 This will destroy the mirror. Reboot again, into single user mode in FreeBSD 9 and resume the upgrade process. Return here to rebuild the mirrors using GPT partitions before the final reboot (or reboot back into single user). Rebuilding GMIRRORThe process is to clear the secondary drive, partition it using GPT, copy the data over to it, boot from this drive, re-partition the first drive similarly, and set up the mirrors for each pat. Be sure to keep the partitions similarly sized to existing ones in the old gmirror, and don't forget any data partition we may have. This is done after booting into the system single-user. A good time is prior to the final reboot of the upgrade steps after the delete-old step or after mergemaster.Set up the partitionsWe need to clear the drive, and partition it. In this example, I set up 512MB root file system, 2G each for swap and /usr, and the rest for /var. dd if=/dev/zero of=/dev/ada1 count=2 gpart create -s gpt ada1 gpart add -t freebsd-boot -l boot-secondary -s 64 ada1 gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada1 gpart add -t freebsd-ufs -l rootfs-secondary -s 512M ada1 gpart add -t freebsd-swap -l swap-secondary -s 2G ada1 gpart add -t freebsd-ufs -l usr-secondary -s 2G ada1 gpart add -t freebsd-ufs -l var-secondary ada1 gpart show ada1 Create new file systemsThe root partition is not recommended to have soft-updates. Also, if you have an SSD, the -t flag is useless once the mirror is set up so there is no point in using that. Since this is an SSD, I'm going to specify the -E flag to clear file system back to the hardware. I also like to label my partitions so my /etc/fstab file is consistent across all machines.newfs -E -L root /dev/gpt/rootfs-secondary newfs -E -L usr -U /dev/gpt/usr-secondary newfs -E -L var -U /dev/gpt/var-secondary Turn on soft-updates journalingSoft updates journaling is a new feature in FreeBSD 9. We enable it now on the new file systems, and should also do it for existing file systems which are not being re-mirrored. tunefs -A -j enable /dev/gpt/var-secondary tunefs -A -j enable /dev/gpt/usr-secondary Copy the dataUse dump + restore to copy the file systems. The -L is probably not necessary since they are not mounted (and in the case of root, mounted read-only).(mount /dev/gpt/rootfs-secondary /mnt && cd /mnt && dump -C24 -0aL -f - / | restore -rf - && cd && umount /mnt) (mount /dev/gpt/var-secondary /mnt && cd /mnt && dump -C24 -0aL -f - /var | restore -rf - && cd && umount /mnt) (mount /dev/gpt/usr-secondary /mnt && cd /mnt && dump -C24 -0aL -f - /usr | restore -rf - && cd && umount /mnt) Reboot into new drivereboot Break into the boot prompt and specify the alternate boot device: set vfs.root.mountfrom=ufs:gpt/rootfs-secondary boot -s This will now be running the system in single user mode from the second drive. Partition the first driveWe partition the first drive identically to the second drive we did above, just using different names: dd if=/dev/zero of=/dev/ada0 count=2 gpart create -s gpt ada0 gpart add -t freebsd-boot -l boot-primary -s 64 ada0 gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada0 gpart add -t freebsd-ufs -l rootfs-primary -s 512M ada0 gpart add -t freebsd-swap -l swap-primary -s 2G ada0 gpart add -t freebsd-ufs -l usr-primary -s 2G ada0 gpart add -t freebsd-ufs -l var-primary ada0 gpart show ada0 Set up the mirrorsgmirror label -v gmroot /dev/gpt/rootfs-secondary gmirror label -v gmusr /dev/gpt/usr-secondary gmirror label -v gmvar /dev/gpt/var-secondary And insert the partitions from the empty drive: gmirror insert gmroot /dev/gpt/rootfs-primary gmirror insert gmusr /dev/gpt/usr-primary gmirror insert gmvar /dev/gpt/var-primary Wait a while for it to finish sync. gmirror status is your friend here.Finishing UpThe file system labels seem to get masked by the mirror labels in this configuration, so you need to update the /etc/fstab to use those.mount -u / mount /dev/mirror/gmusr /usr mount /dev/mirror/gmvar /var Edit /etc/fstab to change the /dev/ufs/XYZ to /dev/mirror/gmXYZ devices for root , usr , and var partitions.Reboot. Profit. |
Mini Blog >