canonical-ci-engineering team mailing list archive

[Thomi Richards <thomi.richards@xxxxxxxxxxxxx>]

Hi,

On Wed, Apr 1, 2015 at 9:16 AM, Celso Providelo <
celso.providelo@xxxxxxxxxxxxx> wrote:

> I think gating releases on pypi is the easiest way to ensure we have
> proper releases and distribution of a library. We could use lp project
> downloads, but that's not supported by pip, which implies and filling
> our pip-caches differently. Pypi releasing is easy and independent
> enough
>
>
Agreed.



> Anyway, I will be really brief about this because the "let's have a
> utils library" thing is already a done deal and as a team we have to
> exercise the right of experimenting. However, bear in mind that this
> topic is already bringing all sort of problems and wastes we agreed we
> would avoid, this discussion is completely side-tracked from what we
> have to deliver next week.
>
> Instead of discussing things that represent direct value to our users, as:
>
> - How to make ADT wrapper/setups more robust ?
> - How to deal with deadletter-ed messages ?
> - How to survive and isolate broken workers ?
> - Can we show results properly in the Dashboard ?
>
>
You're correct, but that's not the full picture. Right now, when we need to
create a new service, there's a lot of infrastructure that has to be copied
around. What's worse, is that when we need to change that infrastructure
code it's a real pain in the ass. Remember when we wanted to change all the
logging handlers to rotate their log files? That took me over an hour, and
multiple MPs, when it should have been a one-MP change. Right now I'm
slowly refining our service code, one service at a time. It's slow, painful
work, and I'd really like to be able to apply that work across all the
services we create.

Another way to think about it is: This code needs to be written; why do you
think putting it in a library costs more than putting it in the service?
Where's the additional overhead coming from?

A good library can, I think, massively reduce the overhead in creating a
new service. My goal is to have to write fewer than 10 lines of code to
create a best-practise no-op rabbitmq-listening uService. Right now it
takes ~ 100 lines.

We are debating new and old ways to create rigid and artificial
> development workflows to lock ourselves in a very dark and lonely
> place where changing code is a burden that should be avoided ...
>
>
That's a total straw-man argument.

I understand that this team had a bad experience with ci-utils, but it
feels to me like your argument boils down to "we had a bad experience with
ciutils, so let's never consider using libraries ever again". Rather, I'd
suggest a more nuanced approach: What made ciutils a bad experience? It
sounds like perhaps backwards compatibility wasn't enforced, and the
library became a dumping ground for anything and everything. We can avoid
those traps, and reap the benefits of not duplicating code everywhere.


> My personal opinion is that I'd rather copy the whole worker module
> and run a full sed renaming for creating a service, and become famous
> in the process as the "CI copy & paste clown", than invest our energy
> and bright brains into something that we already know will end up
> badly.
>
>
Again, that's a straw-man argument. We _don't_ "know [it] will end up
badly".


> Also, let's not forget that we will not be praised if instead of a
> delivering a functional and robust core-image-testing solution, we
> offer a something_utils library to the world.
>
>
Sure - I wouldn't be suggesting this if I didn't think it'd help us
complete the sprint.


> It's an honest report of what I am thinking, and by no means a
> statement that I will not collaborate on this task. Actually, I mean
> exactly the opposite, if we want to do it, let's do it quickly,
> collect the benefits (less c&p) and assess the outcomes in time for
> delivering what we promised in time.
>
>
yup. I'll get this done today.


> Sorry for the ranting ...
>


Not at all - ranting is STRONGLY encouraged, I think this team needs more
ranting, not less of it (y'all are fair too quiet, IMO).

I'll get this done today, and we can assess it's impact at the end of the
sprint. I'm more than happy to abandon this approach if it proves to slow
us down.

Cheers,
-- 
Thomi Richards
thomi.richards@xxxxxxxxxxxxx