canonical-ci-engineering team mailing list archive

[Evan Dandrea <evan.dandrea@xxxxxxxxxxxxx>]

I've managed to get the test-runner to deploy in 213.97s (3.5
minutes¹) from calling juju-deployer, using juju's new btrfs snapshot
support:

 % sudo btrfs subvolume list /var/lib/lxc
ID 262 gen 618 top level 5 path juju-precise-template/rootfs
ID 268 gen 630 top level 5 path evan-local-machine-1/rootfs
ID 269 gen 632 top level 5 path evan-local-machine-2/rootfs
ID 270 gen 633 top level 5 path evan-local-machine-3/rootfs

(If you're measuring this yourself, you'll want to time it off the
second deploy run, as the first will create juju-precise-template, the
lxc container that gets btrfs snapshotted from.)

Setting it up is in theory as simple as putting /var/lib/lxc on a
btrfs filesystem (lxc-clone: true is the default in environments.yaml
in trusty). Unfortunately, I found myself chasing failing to provision
machines for several hours before it finally worked.

Clearing out the following directories seemingly helped:
/var/log/lxc
/var/log/juju
/var/lib/lxc (be sure to use lxc-destroy)
/var/lib/juju
~/.juju/local
~/.juju/environments/local.jenv

It *looks* like we need to move juju-deployer and jujuclient forward,
but not python-websocket:
http://paste.ubuntu.com/7246842/

There's a bug in juju-deployer trunk that breaks on subordinates, thus
the new branch.

I have yet to get this to tear down cleanly, which I think led to the
following pair of bugs:
https://bugs.launchpad.net/juju-core/+bug/1307215
https://bugs.launchpad.net/juju-core/+bug/1303778

To work around that, you'll want to run destroy environment several
times, once for each container:

  % while ! juju destroy-environment local -y --force; do :; done
ERROR failed to destroy lxc container: error executing "lxc-destroy":
lxc_container: Error destroying rootfs for evan-local-machine-1;
Destroying evan-local-machine-1 failed
ERROR error executing "lxc-destroy": lxc_container: Error destroying
rootfs for evan-local-machine-1; Destroying evan-local-machine-1
failed
ERROR exit status 1

Now it's on you to clean up the containers:
  % for x in $(sudo lxc-ls); do [ $x = juju-precise-template ] || sudo
lxc-destroy --name $x; done

¹ I realise this is slightly longer than my aufs deploy (and a full
deploy was a comparable 20 minutes), but I suspect it's down to
VirtualBox overhead. Hopefully you'll see better results.