canonical-ci-engineering team mailing list archive

[Jean-Baptiste Lallement <jean-baptiste.lallement@xxxxxxxxxxxxx>]

Hi Francis,

On 10/30/2013 11:12 PM, Francis Ginther wrote:
> Lets see...
>
> On Wed, Oct 30, 2013 at 8:06 AM, Vincent Ladeuil <vila+ci@xxxxxxxxxxxxx> wrote:
> > This was discussed a couple of hours ago in ##qa and I think this is
> > worth starting the discussion here to keep track.
> >
> > <jibel> vila, I was reading the updates you're doing to the CI doc (https://wiki.canonical.com/UbuntuEngineering/CI/NewRelease). To be clear running otto on another release that its target release is pointless and can only break things more than they are.
> > <jibel> vila, the main purpose of this tool is to run tests on hardware without having to reprovision the system between each run and also have rollback features, ...
> > <jibel> vila, so the kernel on the host and the content of the container must absolutely match
>
> Why, is this an LXC requirement? I've never come across any
> documentation that kernels must match. The reason otto is being used
> is that does the necessary provisioning and rollback needed to test
> packages from a merge proposal and did it quickly by running on
> hardware. Our former solution of running on running on VMs took 3x
> longer and caused more transient test failures.
This is not a requirement of LXC but of the way we use LXC in otto. In 
otto, the container shares several physical devices with the host 
(sound, input, video via mounts of equivalent nodes in /dev and share of 
/run/udev. For proprietary drivers we build the driver inside the 
container that will talk to the kernel module loaded in the kernel on 
the host side) The reason is that we wanted to test the actual hardware 
and isolate processes and changes to the filesystem and get ride of the 
reinstallation pain (which, cumulated, took approximately 2h30 a day and 
was incompatible with the frequency of the daily landing)

If the kernel on the host and user space components inside the container 
do not match (different versions of kernels) there is an important 
chance that it will break or expose weird behaviors.

If you don't need to share physical devices, otto adds lot of complexity 
that may not be required and make the whole system much more difficult 
to deploy for a developer than a bare LXC container. It also requires 
dedicated hardware during the test.
If you need to share physical devices, then the container and the host 
must match.

> > <jibel> vila, why it is forked http://bazaar.launchpad.net/~canonical-ci-engineering/jenkins-launchpad-plugin/autopilot-testrunner-otto-trusty/view/head:/run_otto_job
> > <jibel> ?
> > <vila> jibel: no idea
> > <jibel> vila, it seems to be adding options, and a specific option for local branch
> > <vila> jibel: and given that we cloned the saucy job to create the trusty one, I doubt it's generated
>
> I'm not sure I understand the question. The jenkins job is not
> generated. The job and the testrunner branch do the extra work of
> installing the test packages and PPAs provided by the parent jenkins
> job. No objection to it needing some love, however once are able to do
> this testing reliably on a canonistack/prodstack VM, this whole thing
> can go away.
I understand this will go away once moved to Canonistack. My comment was 
more about adding and patching run_otto_job to another branch rather 
than proposing a patch against trunk. Especially when this option is a 
nice addition to the tool and it was mentioned as missing on 
#ubuntu-ci-eng few days ago.

I'm happy to add you to the otto-dev team if you think it'd be 
easier/faster to process merge requests yourself.

Cheers,

--
Jean-Baptiste
IRC: jibel