canonical-ci-engineering team mailing list archive

[Martin Pitt <martin.pitt@xxxxxxxxxx>]

Francis Ginther [2014-12-08 14:09 -0600]:
> I believe the most immediate answer to "How should a test author declare
> that a particular test suite should only be run on a real phone?" is to
> take the approach that was discussed in Malta [1], and make user of the
> 'Classes' field [2]. Martin, please correct me if I'm wrong, but I believe
> 'Classes' replaced what was originally discussed as tags in that document.

Confirmed.

> Martin, Jean-Baptiste, Vincent,
> 
> For this to work, the current britney service for proposed-migration will
> need to interpret this field and act accordingly. Does britney do this now?

No, as we still use our old infrastructure (alderamin and friends) to
run all the tests -- i. e. there is no choice at all here that britney
could do.

> I'm looking to Martin to help define what these classes should be. For
> example, should we start with just 'phone' as that appears to the testbed
> for which these tests are being developed (this is my assumption)? But as
> Martin mentioned, we don't want to just encode specific platform
> information in the dep8 control file. From our current testing needs,
> 'phone' and 'desktop' are the two classes that first come to mind.

I would try and avoid encoding specific hardware targets. I think it
makes a bit more sense to categorize the nature of the test and
associated packages:

 - graphics: for X.org/Mir drivers and the like, run on machines with all
   major graphics cards
 - hw: for kernel, ubuntu-drivers-common, DKMS etc: run on all our
   supported/certified platforms
 - touch: run on all our supported phones/tables/etc.

So "desktop" might also make sense, but most desktop-related packages
(except compiz/mir) are actually fairly hardware agnostic -- you can
just run them in Xvfb, and thus on pretty much any testbed.

Testbed categorizations *do* make sense for system-wide tests, like
our Ubuntu desktop/server install/upgrade/post-install tests. So we
could have a "desktop" class to request a testbed which isn't a
"minimal debootstrap" type, but a fully installed default desktop
install with running unity etc.

So, I don't think we'll get a perfect categorization of test classes
right at the first shot, but we can start with something simple with
just the two or three classes that we need, and then improve over time.

Martin
-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)

Attachment: signature.asc
Description: Digital signature