Subversion Hell
January 30, 2009 – 10:28 pm
I’ve recently moved my web sites to a new web hosting service. This has mostly been good, but i’ve had one major problem – and it’s my own fault, really.
I’ve been using subversion to keep some of my live sites in sync with my local development system – it makes life so much easier than manually copying files, and it’s essential if you want to keep control of a lot of files in lots of different web sites.
Moving that from one server to another wasn’t much of a drama. But then i discovered a serious security flaw in the way my new hosting service had their subversion service configured. Basically, this flaw meant that anyone could execute arbitrary scripts on the server with root privileges – which is about as serious a security hole as you can get!
It was an effort getting the message over to the sysadmin – i had to explain it in several different ways before they finally clicked that what i was talking about was real and that it was serious. Unfortunately, though, once i had convinced them that it was serious and they’d discussed the bug with the distributors of the particular software packages they use, they decided they should stop all access to the subversion server. This was a bit of a problem for me, as i was quite dependent on using subversion for keeping my live sites in sync with my development system.
Of course, there are other ways of running subversion than connecting directly to the svn server. On my previous hosting system, i’d used svn+ssh – which is subversion over an ssh connection. I tried using this with the new server but it didn’t work. The problem was that the svnserve binary is in /usr/local/bin – but the default path for non-login shells via ssh doesn’t include that directory. And svnsync can’t be configured to use the full path, rather than relying on bash’s $PATH being set properly. It’s very frustrating!
I tried several times to get the sysadmin at my hosting service to reconfigure ssh or jailshell – whichever is responsible – to add /usr/local/bin into the default $PATH, but either they were incapable of understanding what i was asking them or they didn’t want to understand it. I suspect they just couldn’t understand – even though i tried to explain it several times and in several different ways. It’s not a difficult concept to grasp. And, for anyone who’s got a clue, it’s not a difficult problem to solve, either.
Anyway, i knew i could find a way around it if i tried hard enough – and, sure enough, i did. It’s an awkward workaround, but it’s better than manually copying files and trying to keep everything under control.
The way i resolved the problem was to remotely mount my remote server home directory on my local machine, using sshfs. sshfs is a virtual filesystem which works via sftp – you can mount an sshfs filesystem on your local machine in the same way as you can mount a CD, or an external harddrive, or a samba filesystem.
Once i’d got my server home directory mounted on my development machine, i could create an svn repository on it and set it up as a mirror of my local repository, using ‘svnsync initialize
‘. Once it was set up, i could sync the repositories, after a commit, with ‘svnsync synchronize
‘ – as long as the sshfs filesystem was mounted first.
I had quite a lot of trouble getting this all to work – mainly because i had to work out how to make sshfs work in the way the documentation suggested it ought to work by default. The problem was that it wasn’t translating UIDs (user IDs), and the files on the locally mounted filesystem were all owned by my UID on the remote server – which was no good, as it didn’t correspond with my UID on my local machine.
I got round this by using the following filesystem options in /etc/fstab:
rw,nosuid,nodev,max_read=65536,user=will
– well, that’s not exactly what i used, but that’s what mount converted my options into, so i changed fstab to say the same thing as /etc/mtab says when the sshfs is mounted. The reason i did that was because when i try to unmount the filesystem, mount complains with: mount disagrees with the fstab
. I don’t know why it thinks it disagrees, because fstab and mtab say the same thing now, but i still can’t unmount except as root. Doing ‘killall ssh
‘ umounts it though!
Anyway, this is a problem because i want to have a post-commit hook which mounts the ssh filesystem, runs svnsync, and unmounts it again – automatically, every time i do an svn check-in. For some reason, too, when i try that, svnsync doesn’t work either. I’m not sure why yet, as it works fine when i run it manually. It may be that the filesystem hasn’t mounted fully by the time the mount command returns and so when svnsync runs the filesystem’s not there. If that is the problem, i can probably fix it with a ‘sleep’ in between ‘mount’ and ‘svnsync’. I haven’t got round to debugging that part of the process yet. But if i can’t sort out the unmount problem, it’s going to be too clunky to automate anyway, so it won’t be an issue.
So, to recap… I’ve now got to the stage where i can mount the remote home directory on my local system, using sshfs. Then i can run svnsync to update the remote svn repository from my local repository. Now all i’ve got to do is run ‘svn update
‘ on the server to update any changed files in the live sites’ web server documents.
When i tried running ‘svn update
‘ on the server i discovered another problem – the version of svn on the server is older than the one on my development system and it won’t work with the svn repository i’ve created via the sshfs mount! So there was only one thing for it, i had to do the update on my local system via the sshfs mount too. That’s probably the best way to do it anyway, as the sshfs is mounted already anyway and it’s easier than ssh’ing to the server to run the command there.
It’s horribly clunky – and there’s arguably no point in having the sync’d repository on the server, but it’s nice to have as a backup anyway. But at least i can use subversion to keep the live codebase in sync wiht the development system again. Eventually, i hope, the hosting service will sort out the subversion problems and it will all start working properly again. So long as they also upgrade to the latest version of subversion, i’ll be glad of having the repository set up on the server already.
Leave a Reply