canonical-ci-engineering team mailing list archive

[Thomi Richards <thomi.richards@xxxxxxxxxxxxx>]

Hi,


I was going to hold off discussing this till the next sprint in Austin, but
I'm getting frustrated at having to copy and paste significant chunks of
code around from one service to another. We have a a fair amount of code
that help us wrangle the _infrastructure_ behind our services (logging,
kombu, etc).

I have been trying, whenever possible to make sure that that
"infrastructure wrangling" is kept separate from our service logic. In an
ideal world, when writing a new service, the only code I should need to
write is the actual service code itself.

The list of things I shouldn't have to write include:

* Code to make kombu do what we want (given that we want it to do the same
things again and again and again). [1]
* Code to configure logging (log file rotation, logstash etc - again, our
requirements seem standard across all our services here). [3]
* Code to make reporting failed subprocesses easier (such as [2])

I'm proposing that I:

1) Start a new project on launchpad called uservice-utils, which can
contain common utility code (see notes about acceptance criteria below
though)
2) start moving some of this code into that library, at the point at which
it's proved itself to be useful and reusable.

What makes it into the library (acceptance criteria):

* Must be backwards compatible. Our services are depending on this, so
breaking changes won't be allowed.
* Must be reasonably well tested.
* Must not contain ci-specific logic. This is *not* where we store common
data payloads between services (that's a separate library that exists on a
per-sprint basis).

How do we publish & use this?

I propose that we include it in our pip cache for each service. Whether we
actually publish it to pypi or not is open for debate (I don't really mind,
I guess I'm leaning slightly towards publishing it, since it may be useful
for others).

Thoughts? We could easily do this in a day, and the advantage would be that
creating new services becomes significantly less finger-work. We'd also be
able to apply bug-fixes and improvements across the board, rather than
having to manually patch N services at once.

I'd like feedback ASAP, and unless there are strong objections I'll start
working on this tomorrow.


Cheers,

[1]
http://bazaar.launchpad.net/~canonical-ci-engineering/core-image-publisher/trunk/view/head:/core_image_publisher/queue.py

[2]
http://bazaar.launchpad.net/~canonical-ci-engineering/core-image-publisher/trunk/view/head:/core_image_publisher/utils.py

[3]
http://bazaar.launchpad.net/~canonical-ci-engineering/core-image-publisher/trunk/view/head:/core_image_publisher/__init__.py#L52
-- 
Thomi Richards
thomi.richards@xxxxxxxxxxxxx