canonical-ci-engineering team mailing list archive

[Evan Dandrea <evan.dandrea@xxxxxxxxxxxxx>]

I attended this talk on Microservices last night. Stick with it. He's
a bit annoying at the start, but smooths out as he gets into it and so
eloquently explains why I love microservices.

https://skillsmatter.com/skillscasts/5233-separating-fact-from-fiction-what-do-microservices-really-look-like

Intro:

"The next day isn't promised to you. The next hour is not promised to
you. This is relevant to how we build software.

We don't know what we're doing; we haven't got a clue. We have all
sorts of assumptions about what we're building and how it's going to
behave at runtime, and actually most of them end up in conflict with
reality. The problem is, if we buy into [the waterfall way of
modelling software] we assume that reality is the problem. "You've
changed your mind. Don't do that." But if you take a Stoic view:
everything can change, everything is going to be different. I can't
predict any of it.

If we don't know so much, what can we do to prepare ourselves for the
software we're writing such that we are ready for what we don't know?
And that is where microservices start to come in."

----

My notes:

Cloud is a natural pairing for Agile. Cloud is all about being able to
change at runtime.

The first axiom of software development is human comprehension. If we
cannot comprehend our software, we cannot change it. This is about
simplicity, not about being easy. Simple is objective. We can agree on
it, we can measure it. Easy is subjective.

Monolithic services lead to accidental complexity.

You should focus on small, single-purpose services. Stateful or
stateless, it should do just one thing.

Think in terms of data pipelines.

By architecting by services, you can decide on a service by service
basis your scaling strategy.

Optionality is a wonderful property of microservices. The path taken
can be played with by the system at runtime.

Read the Antifragile book¹. Fragile services flake in the face of
change. When people traditionally thought about the opposite of
fragile they thought robust and resilience. These services tended to
be fragile at their hearts and heavily bubble wrapped to make them
robust. The problem with robust and resilience is that at best it
ignores change. When shit happens, it can deal with it because largely
it can ignore it. The alternative is that potentially it can resist
your change. This is flawed. It also implies perfection. Enter
Antifragile:

Antifragile doesn't try to ignore change, doesn't try to avoid change.
It gets better with change.

Chaos Monkey stresses fragility to make the system *better*, not just
to test if the system survives. It's like muscles. Under the right set
of repeated stresses it will improve, just like exercise.

On traditional SOA: We're not optimising for pure modularity. Most of
the systems he's encountered are not optimised for change, they're
optimised to break things down into a hierarchy. We need to support
change. I need my system to be able to adapt. Create peer services,
not hierarchies.

Polyglot. You want to hire engineers based on their abilities, not
whether they have skills in a particular language.

What's the job of an architect? To make the big risky decisions.
"We've got to use Java across the entire organisation." With
microservices, those decisions get diluted a lot. Each service is an
opportunity to decide how we implement it.

Microservice architectures tend to be very flat. Traditional SOA has
big frontends, nitty gritty down at the bottom. This was almost an
object-oriented hierarchy. With microservices we tend to think more
functionally, more single purpose. A functional pipeline. A flat array
of services that explicitly or implicitly interact with each other.

What's the downside of microservices? Ben Wootton has an extremely
good counter article on microservices called "Not A Free Lunch²." The
stress placed on devops is significant. It's no free lunch...yet.
There are plenty of people in the industry trying to make this much,
much simpler. Oh hai CDO.

On this devops challenge, specifically on monitoring, don't have a
billion services all saying "I'm okay, I'm okay, I'm okay, I'm dead,
I'm okay." They key abstraction for this is the pipeline. The pipeline
knows when things start to go wrong. Services themselves you try to
keep as extremely simple as possible, but the pipeline as well as
trying to work with a number of services will also know when things
are fundamentally becoming an issue.

Is it SOA? Yes, but it's not SOA as we've traditionally known it.

The handoff is in the pipeline. The pipelines are nothing more than
microservices in their own right. They just have the single purpose of
working across one layer of abstraction across a number of services.

Focus on the data flow. This is the same conversation that's happening
around big data. We're
starting to move towards a sensible, ubiquitous language for building
software, which is around the manipulation of data. It's not about the
structure of software, but rather its utility.

So focus on data flow, rather than getting bogged down in structure.

Create an environment to experiment, try new things. This is easy in
microservices. At the most basic level you're free to chose whatever
languages and tools you want.

Notes from question and answer portion:

The "share nothing" concept of software architecture is raised. Don't
try to share state, ever.

Thinking in data flows. The first casualty of a microservice
architecture is a shared database. Each service maintains its own
state. Instead of having central state, emit this information out so
you get options on where you store it and how its handled by the
services.

Rather than communicate where it needs to happen, communicate *what*
needs to happen. Event-driven models like what nodejs provide are good
for this.

Use circuit breakers to help ensure that you're not heavily coupled temporally.

You can get a half-off discount on his book³:
https://leanpub.com/antifragilesoftware/c/london-user-group

There will be a microservices conference in London in November (early
bird: £200 + VAT):
https://skillsmatter.com/conferences/6312-mucon

¹ http://www.amazon.com/Antifragile-Things-That-Disorder-Incerto/dp/0812979680/ref=sr_1_1?s=books&ie=UTF8&qid=1402671731&sr=1-1&keywords=antifragile
² http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html
³ Which you're welcome to expense, if you report into me.