Back from The Great Strugglers Boat Trip, 2007

Crossing the Pontcysyllte Aqueduct
I’m back now from our week-long narrowboat trip from Middlewich in Cheshire to Llangollen in Wales and back again. I had a fantastic time and was really happy to get away from it all; pity it’s back to work on Monday.

I think everyone else had a good time too. Despite a couple of rainy days I think we were fairly lucky with the weather on the whole and we enjoyed a nice and sunny first trip over the aqueduct.
In a lock
The boat we hired, Poplar, was designed for a maximum of 12 people but that would be with all 8 bunks in use and the two dining tables converted into double beds. As it was with only 5 of us lads on board the boat felt quite cramped. We needed to use some of the bunks for extra storage space as well. I think the boat could have accommodated one more person (a total of 6) with relative comfort, or a total of 8 people if they were willing to be rather cramped. I just can’t imagine it with 12 people, even if some of them were children!
Matt and Popey in the lock
At 70ft in length I was a bit nervous about driving it, as it’s quite hard to even see as far as the front end of it! I didn’t really have any need to be concerned though; it wasn’t really any more difficult than any of the other boats we have hired. They’re pretty indestructible things anyway and your biggest concern is slamming into someone else’s boat and scraping their paintwork.

I was probably in greatest fear for my life while standing on top of the boat while going over the Pontcysyllte Aqueduct. I had to sit down in the middle as the wind was blowing a gale to the point where it was actually dangerous. Next scariest thing was when Tim decided to stop the boat by just looping the middle rope over a mooring post while it was still going at a fair speed. It tightened up and dragged the boat over at an alarming angle so it felt like the whole thing was going to tip over and capsize. Popey fell over and so did a lot of the contents of the boat!

Welcome to Wales
The wiki page covers the plan in excruciating detail, though we didn’t stick to our suggested stopping points very well since we thought it best to push on further some days and take it easy at other times.

My photos are in my gallery and Popey has his online now too.

If your Ubuntu Gutsy chroot/domU segfaults weirdly under Xen…

If your Ubuntu Gutsy chroot/domU segfaults weirdly under Xen, maybe you forgot to move /lib/tls out of the way?

I encountered this one yesterday as one of my customers asked me about problems debootstrapping under Xen. I have done several debootstraps before so wasn’t aware of any obvious problem, and this irritated me enough to make me delve into it further.

It was actually the dpkg --configure <package> step done for each package after debootstrap that was failing with a segfault. Having a look at the post-installaton script for that first package, the part where it was segfaulting was when it tried to do:

update-rc.d module-init-tools start 15 S .

a quick strace of that revealed:

open(".", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 3
fstat64(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
fcntl64(3, F_SETFD, FD_CLOEXEC)         = 0
getdents64(3, /* 11 entries */, 4096)   = 368
getdents64(3, /* 0 entries */, 4096)    = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---

Seeing how low-level this had become I started to consider where the bug would have to be, and soon thought “what about libc?” There are well-known issues with 32 bit Xen and the TLS features of GNU libc, which is why Debian has provided a libc6-xen package since Etch. Sure enough, doing:

# mv /lib/tls /lib/tls.disabled

made this problem go away and resulted in a perfectly usable chroot environment.

TLS is normally one of the first things I check when doing things under Xen, but I didn’t think for one minute that Gutsy would still be shipping with a non-fixed libc.

I’m not really an Ubuntu user and am not aware of their packaging strategies for Xen stuff but I was also surprised to see that there is apparently no libc6-xen package in Gutsy. What is the recommended way for Ubuntu domUs to deal with this problem?

Moving the directory out of the way works, but it sucks as anything but a workaround as any libc update will replace the files. That happens to my users with older distributions all the time and they only find out because of the masses of warnings in the kernel log. Fair enough, but if it’s going to start causing weird segfaults as well then I don’t really want that support burden.

(Update: it was only because I didn’t have all the repositories in my /etc/apt/sources.list. Gutsy does have a libc6-xen package.)

It is perhaps time for me to increase the priority of moving to 64bit. (most of my servers already do 64bit but I run them 32bit as some of my customers’ chosen distributions have iffy 64bit architecture support — that may not be the case any longer)