Slicehost’s Rescue Mode rocks; Gentoo’s Xen guest /proc /sys mounting bug

This is a tutorial / fan post for Slicehost’s Rescue Mode. Rescue Mode saved me from my own stupid mistakes and also a known Gentoo bug.

While fiddling around in my /etc/fstab file (added “data=writeback” to the mount options), I rendered my Gentoo VPS unbootable. Oh well, I just need to boot a LiveCD and fix my stupid fstab. Wait, this isn’t a physical Gentoo box in front of me, but a virtual server hosted miles away from me…

Ta da, Slicehost’s Rescue Mode to the, uhh, rescue. I log into SliceManager, go to the Rescue panel of my slice, and click “Enter Rescue Mode”. SliceManager gives me a temporary root password, and finishes building the Rescue Slice in a few minutes. Instant LiveCD-like rescue environment!

To fix my fstab, I SSH into my Rescue Slice with the temporary root password. The partition holding my regular slice is available as /dev/sda1 (and my swap partition is /dev/sda2 if I need that for some reason). I mount my regular slice,

mkdir /mnt/slice
mount /dev/sda1 /mnt/slice

fix my fstab file,

nano /mnt/slice/etc/fstab

and finally return to SliceManager and click “Exit Rescue Mode”. Problem solved. Slicehost’s Rescue Mode rocks!

Unfortunately, my VPS failed to boot again but with a different error. It died upon trying to mount /proc.

The "mount" command failed with error: proc already mounted

It turns out this is a known Gentoo bug when working with Xen. Apparently Gentoo Xen guests should not try to mount /proc and /sys. I don’t understand what triggered this problem or why I did not encounter it before.

So back I go into Rescue Mode to fix this by commenting out the section of /sbin/rc that attempts to mount /proc and /sys (if you only skip /proc mounting, the next bootup will die trying to mount /sys).

mkdir /mnt/slice
mount /dev/sda1 /mnt/slice
nano /mnt/slice/sbin/rc

Around line 217, comment out these chunks of code like so: (source)

#       check_statedir /proc
#
#       ebegin "Mounting proc at /proc"
#       if [[ ${RC_USE_FSTAB} = "yes" ]] ; then
#               mntcmd=$(get_mount_fstab /proc)
#       else
#               unset mntcmd
#       fi
#       try mount -n ${mntcmd:--t proc proc /proc -o noexec,nosuid,nodev}
#       eend $?

and a few lines later:

#       if [ "$(get_KV)" -ge "$(KV_to_int '2.6.0')" ] ; then
#               if [[ -d /sys ]] ; then
#                       ebegin "Mounting sysfs at /sys"
#                       if [[ ${RC_USE_FSTAB} = "yes" ]] ; then
#                               mntcmd=$(get_mount_fstab /sys)
#                       else
#                               unset mntcmd
#                       fi
#                       try mount -n ${mntcmd:--t sysfs sysfs /sys -o noexec,nosuid,nodev}
#                       eend $?
#               else
#                       ewarn "No /sys to mount sysfs needed in 2.6 and later kernels!"
#               fi
#       fi

That’s it. Happy rescuing.

If you liked this post, please subscribe to my feed. Thanks for visiting!

Tummblr