canonical-ci-engineering team mailing list archive

[Vincent Ladeuil <vila+ci@xxxxxxxxxxxxx>]

>>>>> Evan Dandrea <evan.dandrea@xxxxxxxxxxxxx> writes:

    > Vincent just debugged an issue to being caused by a patch someone made
    > to live code on jatayu.

To give credit where it's due:

- didrocks gave hints that a patch related to the issue has been seen at
  some point in the past but he couldn't remember in which form nor
  where it was applied (and to be honest, I think nobody suspected that
  patch was applied to an installed package[2]),

- fginther came up with a reproducing recipe that was succeeding on
  magners but failing on jatayu (you know, the basic definition of a
  test that fails first ;)

All I did was being stubborn enough to keep searching why that test was
succeeding on magners (and even there fginther shared his knowledge and
didn't let me follow wrong tracks).

The key thing I learned here, was that Francis had a focused failing
test.

This is a wonderful example of how *I*[1] practice TDD: design a failing
test first. It doesn't have to be automated at that point but it has to
be reproducible and as focused as possible because it will need to be
run repeatedly until the issue is diagnosed.

At that point, Francis had the knowledge, mine was limited (and roughly
still is) to: Francis knows how to reproduce hence check potential
fixes.

The next step is to share that knowledge so it spreads.

It's not always worth sharing though, but in this specific case, that
test captures a feature: cu2d needs to create a MP and submit it to
launchpad to merge back a validated change. A key piece of the ci
infrastructure.

If this test succeeds, the feature is supported. If it fails, we have an
issue.

We want that test to be automated :)

And we want that test to be ~10 lines long.

Of course, today, setting up the test environment cannot be achieved in
10 lines, we need to have a test infrastructure that give us enough
helpers to make that possible.

In the mean time, as long as we capture a description of that test that
anybody in the team can use to reproduce, we're good ;)

        Vincent

[1]: I won't ever pretend it's the True Way or even the right one, but
     that's surely a key part: sharing knowledge. Tests are good at
     capturing knowledge in a way that anybody can reuse.

[2]: I don't really care who did it either. As long as we stop doing
     that without sharing the knowledge (err, no I mean, we should just
     stop doing that ! Well, except we did it in the same way on jatayu,
     oh well ;)