← Back to team overview

canonical-ci-engineering team mailing list archive

Re: Autopilot Testing Sustainability

 

Le 15/03/2015 20:39, Thomi Richards a écrit :
Hi,

On Fri, Mar 13, 2015 at 8:24 AM, Ursula Junque
<ursula.junque@xxxxxxxxxxxxx <mailto:ursula.junque@xxxxxxxxxxxxx>> wrote:

    So what solutions might be possible?


Sorry for the delay in getting to this.


To be honest, I don't like any of the proposed solutions so far - they
all feel like ugly hacks to me. If we're going to spend energy solving
the problem, let's make sure we solve it well :D

The root problem, AIUI is that while we can retrieve old images, the
tools we use to test those images aren't similarly retrievable.

There's several different conceptual angles we can try to solve this
problem:

1) We could try and change the Ubuntu archive so it keeps old versions
around forever. Almost certainly not going to happen.
2) We could try and change the phone images so they include the test tools.

#2 seems like a much more feasible goal to me. Has anyone looked at the
possibility of including the test dependencies in the phone image, but
disabling them when not in developer mode? I can anticipate several
concerns with this approach (using up valuable image space for one,
security for another), but I think we can address those concerns as they
arise.


Thoughts?
Just a thought, not sure how it can be implemented. For each image built and published to s-i.u.c, would it be possible to build a custom tarball containing the test tools and apply it on top the image?

This custom tarball would be built at the same time than the image and would use the same version of the archive (there is a small risk of race when a package is updated in the archive between the time the rootfs is built and the time the custom tarball is generated)

The advantages would be:
- It exactly matches the version of the image
- You can keep the same history than the images available on s-i.u.c
- It is installed only when testing an image (no impact on production images)
- You can load all the required dependencies for your tests.

There are a couple of technical points to solve (how to identify the right test deps automatically, scripted way to apply a custom tarball on top of an already customized image, what is the impact on testing of tainting the image this way, ...) but it sounds feasible.


JB.

--
Jean-Baptiste Lallement
Canonical Services Ltd.
IRC: jibel


References