canonical-ci-engineering team mailing list archive

[Alexander Sack <asac@xxxxxxxxxxxxx>]

On Thu, Jul 3, 2014 at 10:52 PM, Thomi Richards
<thomi.richards@xxxxxxxxxxxxx> wrote:
>
> On Thu, Jul 3, 2014 at 8:21 PM, Alexander Sack <asac@xxxxxxxxxxxxx> wrote:
>>
>> I assume
>> http://ci.ubuntu.com/smokeng/utopic/touch/mako/109:20140702.1:20140625/8832/health-check/
>> would be a good example of what could feed into that system?
>
>
> Exactly, and we already have a work item to add that as soon as it's
> deployed.

AWESOME.

Related to this topic (and I am not sure where else to seed this wish,
so I am asking QA to keep an eye on this): whatever measurement based
test we plan to implement, can QA also keep an eye on ALWAYS doing
those so we get a good grasp on the variance of the result we measure?

Looking ahead I am already plagued with how we can even use
measurements likethis to effectively gate and drive engineering given
how hard it is for us to do that effectively for simple black and
white true/false tests.

In other words: for measurement based tests with variance, flakiness
will be the theme everywhere and only by having very low and
reproducible variance levels I will even be able to talk to
engineering teams and managers about buying into the idea to gate and
identify promotion blockers out of those.

Thanks for remembering that if anything related to implement
measurement based tests for our CI comes on your plate.


>
>
> --
> Thomi Richards
> thomi.richards@xxxxxxxxxxxxx