canonical-ci-engineering team mailing list archive

[Vincent Ladeuil <vila+ci@xxxxxxxxxxxxx>]

>>>>> Thomi Richards <thomi.richards@xxxxxxxxxxxxx> writes:

    > Hello CI-ers,

Hello dear QA-er !

    > We spoke about this in DC, but I wanted to circle back and pick up this
    > conversation again.


    > I'd like to add a suite of deployment tests for NFSS. It seems that I need
    > to add a test suite in the 'tests/' directory, but I'm missing a few bits
    > if information:


    >    - I assume these tests are run during merge proposal CI phase?

Yes.

    >    - I assume I should just run './run-tests' to run the tests? If I do
    >    that on utopic I get 5 errors (http://pastebin.ubuntu.com/8976623/) How
    >    can I fix these?

You're supposed to run under a venv so utopic shouldn't be an issue (but
I'm still on trusty so I can't easily check).

That being said, note that run-tests will create a temp venv if it
detects that it's not under one already (note the /dev/shm/venv-g_5hl0/
paths in your tracebacks). So I would advice that you create one that
you can reuse instead, and go there before running run-tests.

Now, for the errors themselves... it's weird... as a wild wild guess I
would imagine that you have the bzr check_signatures option set ?

If that's the case, it's a test isolation bug as those tests should
isolate themselves from your home directory.

A workaround would be to disable that bzr option and re-run the tests to
confirm.

    >    - I assume I need to direct the test system to deploy NFSS before
    >    running the tests? If so, how do I do that?

You deploy nfss before running the tests.

Except that many tests will break because they assume that uci-engine is
deployed ;)

    >    - Once NFSS is deployed, how do I get the config information to tell me
    >    *where* the nfss components (restish, postgres etc) were deployed to?
    >    (i.e.- ip addresses & tcp ports). I note that some tests import 'deploy',
    >    which seems to contain commands to get the juju status, but I can't see
    >    where that's coming from in the source tree?

Ha. The mythical unit_config in the branch root.

For hysterical raisins, this is badly handled right now but has been
identified as a pain point whose importance you just raised by running
into it ;)

Long story short: that file is created as part of running the tests
(instead of each test needing it creating its own private version).

    > I realise that's an awful lot of questions. To sweeten the deal, I'm
    > proposing to write documentation for this process, as I learn it - the docs
    > in README are a good start, but I think it could use a little expansion.

Patches welcome !

        Vincent