If your Ubuntu Gutsy chroot/domU segfaults weirdly under Xen, maybe you forgot to move /lib/tls out of the way?

I encountered this one yesterday as one of my customers asked me about problems debootstrapping under Xen. I have done several debootstraps before so wasn’t aware of any obvious problem, and this irritated me enough to make me delve into it further.

It was actually the dpkg --configure <package> step done for each package after debootstrap that was failing with a segfault. Having a look at the post-installaton script for that first package, the part where it was segfaulting was when it tried to do:

update-rc.d module-init-tools start 15 S .

a quick strace of that revealed:

open(".", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 3
fstat64(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
fcntl64(3, F_SETFD, FD_CLOEXEC)         = 0
getdents64(3, /* 11 entries */, 4096)   = 368
getdents64(3, /* 0 entries */, 4096)    = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---

Seeing how low-level this had become I started to consider where the bug would have to be, and soon thought “what about libc?” There are well-known issues with 32 bit Xen and the TLS features of GNU libc, which is why Debian has provided a libc6-xen package since Etch. Sure enough, doing:

# mv /lib/tls /lib/tls.disabled

made this problem go away and resulted in a perfectly usable chroot environment.

TLS is normally one of the first things I check when doing things under Xen, but I didn’t think for one minute that Gutsy would still be shipping with a non-fixed libc.

I’m not really an Ubuntu user and am not aware of their packaging strategies for Xen stuff but I was also surprised to see that there is apparently no libc6-xen package in Gutsy. What is the recommended way for Ubuntu domUs to deal with this problem?

Moving the directory out of the way works, but it sucks as anything but a workaround as any libc update will replace the files. That happens to my users with older distributions all the time and they only find out because of the masses of warnings in the kernel log. Fair enough, but if it’s going to start causing weird segfaults as well then I don’t really want that support burden.

(Update: it was only because I didn’t have all the repositories in my /etc/apt/sources.list. Gutsy does have a libc6-xen package.)

It is perhaps time for me to increase the priority of moving to 64bit. (most of my servers already do 64bit but I run them 32bit as some of my customers’ chosen distributions have iffy 64bit architecture support — that may not be the case any longer)