canonical-ci-engineering team mailing list archive

[Martin Pitt <martin.pitt@xxxxxxxxxx>]

Hello again,

Steve Langasek [2014-12-10  0:04 -0800]:
> I think that provides most of the guidance we need.  The only thing I'm
> still not sure about is the "fail gracefully" part.

I thought about this again. I don't think we should put too much
effort into this. It's much more appropriate and also easier to teach
Britney (or our test runners) to just skip tests that declare Classes:
that we can't currently handle.

> All of the interfaces invoked by our tests are things that, under
> DEP8, would be declared as dependencies of the tests (AIUI), and
> there's no interface for querying the key bit that will make the
> test work (i.e., "is the recovery partition set up to upgrade us,
> and does this device support rebooting into recovery mode?").  So
> for the time being I don't think we can be very graceful.

Agreed. And I think that's not too bad -- there might be reasons why
you want to locally try and run a test on a different kind of testbed
to see how far you get, we should not outright disallow that. We of
course need to make sure that we run on appropriate testbeds in our CI
infrastructure and provide the testbed classes that we need.

> Also, I saw "touch" suggested in this thread as a class name.  What
> semantics do we expect that to express?  What I need to express is
> "this test needs to be run on a real device, because that's the only place
> we can reboot into the recovery partition".

Then I think we should say just that by using an "adb-phone" or
"adb-touch" class, only have actual phones/tablets (and maybe the
emulator one day once it learns about that) in that class.

> I'm not sure that's the obvious semantic of "touch"; I think we
> should agree on what we're using these classes for before we start
> declaring them, otherwise I might as well not declare the class at
> all in the near term since there's nothing that can consume it
> meaningfully.

Full ack.

Martin

-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)

Attachment: signature.asc
Description: Digital signature