These are the slides from my Continuous Lifecycle London keynote.
Continuous integration. Continuous deployment. Continuous delivery. We’re moving as fast as we can, but are we going in the right direction? In this talk, we’ll discuss why “But why?” is the most important question you can ask along the way, and look at some of the detours and roadblocks that can hinder your journey: vestigial infrastructure, migrations that never end, and the kinds of technical debt that can DDoS a whole organisation.
More than that, we’ll ask the big questions: Why are we doing any of this? Why are we all here? What are we for?
By adding continuous introspection (and a little existential angst) to the continuous lifecycle, we can go faster, better.
We live in a busy world. There is a lot we can choose to do. And right now you’ve chosen to come to a conference (or you haven’t, but you’re just reading this slide deck! Thanks for being here either way!).
I think that can be difficult for a lot of us to step away from whatever we're creating or whatever we're debugging, and do something else for a couple of days. Disconnecting feels weird. It feels unproductive. But this is a talk about being deliberate. Sometimes you need to deliberately slow down a bit so that you go faster in the long run.
It's unusual in our industry to say to slow down. We like going fast! We are all bias towards action, agile, move fast and break things! We call things names like Swift and SPDY and Velocity. We put our rubies onto rails!
And this conference is called Continuous Lifecycle. We are all about moving, all about speed.
And, let's be clear, that's good. According to...
...the State of DevOps report (which I recommend reading) high performing teams do move quickly. The report says that highest performing companies deploy 46 times as often as the lowest performing. They can push in under an hour and roll back really quickly.
There's a ton of advantage and flexibility to being able to move quickly. You can respond to things your customers want. You can take small risks, secure in the knowledge that you can recover quickly if there's a mistake. It's great to be able to go fast. So long as you’re going in the right direction.
But if you’re going in the wrong direction, going fast just makes you wronger. As Peter Drucker famously said..
…there's nothing so useless as doing efficiently that which should not be done at all.
Sometimes we need to slow. down. and think really hard about where we're going and what we're doing on the way. Sometimes we need to step outside the continuous: we need to do things in a slower and less obvious way, so that we go faster and better in the long run.
Hello! My name is Tanya and I'm a principal software engineer at Squarespace in New York. We have offices in Portland and Dublin as well by the way (and we’re hiring everywhere) and if you're in Dublin we host a great semi-regular SRE meetup.
I'm whereistanya on twitter and github, and I blog about tech and organisational stuff here at noidea.dog, which is, of course, a Squarespace site :-)
And I really love going places on trains. The last time I was in London, for example, I was actually on the way from Dublin to Glasgow.
I flew in to Heathrow, had a bit of extra time so I went to see the Magna Carta at the British Library for an hour, and then I got on the Caledonian sleeper (which is a lovely train, by the way).
You might point out that Dublin to Glasgow, is a direct flight which is shorter than the flight from Dublin to London and much shorter than the train from London to Glasgow, so a reasonable question would be "why?" and the answer of course would be...
I do that a lot: I go somewhere out of my way just so I can spend a bunch of time on a train. It's so peaceful. I love the way it forces me to take a bunch of time where I can't do my regular, continuous day to day things. But it's not downtime.
On trains I am more productive: I write better, I code better, I can think thoughts I just wouldn't have had otherwise. All because I'm forced to slow down.
Going to Glasgow by train via London took a bit longer. But I still arrived at the place I needed to be at by the time I needed to be there, I got more done on the way, and I got to see the Magna Carta.
Ok, why am I talking about trains. A lot of the talks at any conference are about how to go fast. (And, again, that's excellent and you should do that.) But this one's going to do the opposite. We're going to talk about some things we only get to do if we go a little slower. We're going to think outside the continuous and outside the quickest way to get to somewhere.
The talk's not going to take the most direct route either. We'll wander a bit and take some diversions. We'll still get where we're going on time, but hopefully see some interesting things along the way.
Here's a rail map for where we're going to go today.
We're going to start by figuring out where we're even going. Where do we want to be? How do we think about that?
Then we'll get distracted by something shiny, which will seem to get us closer to our goal, but ultimately slow us down.
Then we'll stop and look at a few things that we can only do if we take a little longer to get where we're going, little station stops that slow us down in the moment, but that ultimately make us faster, and happier.
We'll get very briefly existential… it will be alarming! But we will find our way back out of there, and finally arrive at a destination of some sort! Which we'll choose in a minute when we ask….
…what do we want out of our work? Where do we want to go?
So I work in infrastructure, and I'm sometimes jealous of my friends in product type organisations, because I feel like this is well covered for them.
Product managers say "We should make this thing!" and then everyone works together to make the thing and later the thing launches and there's an ice cream party to celebrate the launch.
(If product orgs don't work exactly like that, don't spoil my illusions by telling me.)
We usually don't really have product management in infrastructure or SRE or release engineering or other orgs who are mostly facing internal customers, and I think it's a huge loss. We don't have this regular powerful check in for "Who are our customers? Are they happy? What do they actually want?"
Infrastructure goals are often more like "just… continually make everything better?". And continuous improvement is great, but it's hard to feel like you ever get to a party type of a place.
In the absence of product management, I think it's worth figuring this ourselves. Where do we want to get to? What are big goals we can choose for ourselves?
There's this metaphor I like. Remember the game Civilisation? It was released in 1991 and has had lots of different releases since then. The idea was that you would be a civilization, and you'd start at prehistory with very primitive skills and technologies. But as you'd go along, each turn you'd get to build things or improve your cities or interact with other civilizations.
There were a bunch of different ways to win, like you could win by military conquest -- you'd kill everyone who wasn't on your team -- or you could win by building a spaceship, which I think is a much cooler way to win at society.
The whole way through the game, you'd get to research technologies.
But you couldn't just jump ahead and research spaceships, you'd have to first of all learn sailing and pottery and the mysterious and secret art of writing stuff down and so on; Then you could build on that until, near the end, you'd research space travel. I haven't played in a long time so I don't know what the research tech tree is like in recent games. You can probably research blockchain and, like, serverless now, I don't know.
I always think of that tech tree when people talk about the stuff they want to do. Because a lot of the time, you're not working on a project for the sake of the project. You're working on it for the thing it unlocks.
Say you’re working on Physics. Physics is cool, you know? You drop things, they fall down. How fast does it fall down? Wooo let's find out with physics! But it's hard to explain to someone who doesn't already think it's cool. "Ok, but how does it make my life better?" "PHYSICS!"
But if you tell them you need physics so that you can get steam engines...
...well that's a bit more concrete. And if you tell them your actual goal is to build a railroad…
...that's easy to understand. (If you don't like railroads you're like an alien to me, but that's fine. You do you. Build Atomic Theory or something instead. Just choose a goal.)
But the thing is, if you're building a railroad, you can't just do physics. You also need...
…bridge building and invention and engineering and a bunch of other stuff. If you just do physics, you don't get a win.
If your ultimate goal is to build a railroad, you need everyone believing they're building a railroad and doing all of the work necessary to get you there. Otherwise, you end up with a whole lot of partially done things.
You have 70% of a railroad but you can't go on trains yet, and 40% of metallurgy but you can't make steel. It's just a whole lot of things in flight. You will eventually get one of them finished, but you might like to go somewhere now, not in a few years when you happen to get there.
As an individual or a group, it's worth stepping back and making a model of your civ tech tree and choosing a bunch of goals.
Not just OKRs, or quarterly roadmaps or whatever, but big long-term destinations. I've heard this called "the North Star" or "the shining city on the hill". This sort of thing should mostly come from the organization or the company. I think everyone in an org, ideally everyone in a company, should be aiming towards the same place.
But teams can choose their own north star too. You can pick a time, maybe two or three years from now, and write down what you would love your world to look like then. Describe the ideal. Describe what you'd want to have, if you didn't have something already. And then share it broadly. People might argue with you, but we'll talk later about why people arguing with you is actually a good thing.
And only then, when you have a really solid idea of where you want to go, then start thinking about how you would get there. If you start out by looking at where it's possible to go, you're going to limit your options. You need to be able to choose places you can't see from where you are.
This sort of thing takes time.
But it also saves time.
It's the difference between choosing a restaurant and telling people you're going to meet there at 8, vs walking around a city with a bunch of hungry people and stopping and talking at every corner about whether this street or that street has good restaurants.
I love having big organisational goals to aim at, and then everyone figuring out their own way to get there. It gives people a ton of autonomy, and it turns out we like that. Let's talk about human motivation!
If you do a search for motivation on Amazon it offers you more than 50 thousand books and I have read none of them. Instead I opened the wikipedia page on Motivation and it was 39 pages long and I had things to do so I didn't read that either. Instead I will just tell you the two theories of motivation that you almost certainly already know. (This is the kind of full service keynoting I am offering here.)
In 1943, Abraham Maslow wrote this paper called "A Theory of Human Motivation". He laid out this hierarchy of needs: we have physiological needs first -- food and air and broadband internet and so on -- and if we have enough of that, we look for safety, then friendship and belonging, then esteem of ourselves and others and then self-actualization.
And then in 2009, Daniel Pink built out the "self-actualization" part of that. He says that when it comes to creative work needing decision making, human motivation comes from three things:
Autonomy, which is our desire to choose what to work on.
Mastery, which is our desire to get better at something.
And purpose, which is the desire to do something meaningful and important.
We're self actualized when we can choose our own work, do it in a way that builds skills, and feel like we're going towards an important goal.
So there’s our destination. We want to get to a goal we’ve chosen — here I’m saying we’ve decided to build a railroad -- and we want to have autonomy, mastery and purpose as we do it.
Ok. How do we get there?
Well, there's a way that is very attractive, which is that everyone just works on whatever they think is most important. But, however we try to be impartial about it, often that's heavily influenced by whatever technology we think would be fun to work with.
I've often heard people calling this sort of thing resume-driven development, but I think that's too cynical.
I call it shiny-driven development. I think most of the time, we're genuinely interested in learning and doing something cool. I don't think we're just packing our resumes. I think we are really drawn to shiny problems we can fix.
Our technology is continuously getting better, which is great, but it's also difficult. Every day some new, objectively better thing becomes available. Our best practices change really quickly.
And when we look around, we can see so many interesting problems to solve. It's hard to avoid them.
I think this is even more true for those of us who come from SRE kinds of backgrounds, or who are used to running things in production. We're wired to take disordered things and sort of absent-mindedly put them in order as we walk past.
When we see things we can fix, we want to fix them. I mean, I like that about us! We're motivated by making things better.
And the industry encourages this! Performance evaluations often use this word, "impact". What impact did this person have? What did they do? So everyone wants to show the excellent things they did, the problems they solved, the projects they owned.
Which can be a problem, because we're supposed to be on teams.
In her article, the Origins of Opera and the Future of Programming, Jessica Kerr talks about generative vs productive work. She says productivity is what one human does. Generativity is the improvement they create.
If you can make ten people each much better, you clearly have great generative impact. But we don't always reward that the way we reward one big glorious productive success. And so the pressure is on every individual to show impact and individually do a great thing, and you can end up with a team of six people doing six (or seven) things at once because everyone needs to lead something.
So we're really both internally and externally motivated to follow the shiny track. But it has a bunch of downsides.
One is that we might not solve the most important problems.
If I have five low-priority things I've chosen to do and you have five high-priority things and we both do our most important one, we get the 1st and the 6th most important thing done. Maybe I should have worked on your 2nd most important one?
And if you have something that's the most important thing on the planet, but it depends on help from me, it can be very hard for us to get something finished. Especially if I'm more interested in doing some big project I think will show my individual impact. This is how we end up with 70% of a railroad and 40% of metallurgy!
Or maybe the thing I'm doing is duplicating work that someone else is doing. Or maybe I'm solving a problem that's about to be completely obsoleted by some other change.
Or maybe the project I've claimed is too big for one person.
You know the thing where you decide to clean something up -- a thousand Marie Kondo memes on the internet convince you to finally clean out your clothes closet or something — and after three hours you're starting to run out of steam but you have an incredible mess -- much worse than you started with.
Software projects that are chosen for being shiny really often go like that. We start strong. But after a while it gets hard to make progress, there are some edge cases that can't work… and it's a lot more difficult and less shiny now. And we tell ourselves we're definitely not giving up and will come back to it juuust after we take care of this other shiny thing we can see that's a little easier, a little shinier.
And maybe we will come back to it! That's the mystery of the half-migration, the Schrodinger's cat migration: it can go either way. Is it alive? Is it dead? Nobody knows!
So everyone has to deal with two separate futures: the future where you get around to finishing the thing, and the future where you don't. If someone's building a new service, they have to second guess what's going to happen, or support both the old way and the new way.
In the meantime, it's like you've taken a lock on the project. You've claimed it for yourself. it would be much more work now for someone else to solve the problem you set out to solve, because first they'd need to clean up your mess. So it's going to stay half-done for a long. long. time.
It's like you've just added another long-running process to your org's process table. Which is maybe ok: teams and orgs can handle a lot of processes. But only if they eventually get closed. Because what happens is that other projects get blocked waiting for you. Maybe they can't make a decision until you make a decision. Maybe they don't want to solve a problem you're about to obsolete. Maybe they need some resource you're hogging, usually people who've been at the company a long time who end up being bottlenecks. So they wait. And then other projects get blocked waiting for them.
Over time you get into a weird state where even though it doesn't feel like very much is getting finished, everyone is busy, and everyone is waiting on everyone else. The system starts feeling sluggish: it's much harder to get things done than it used to be.
Though, you know, I'm describing organisational load here as if we're all processes on a single server. But we're more complicated than that. Organisations are more like a messy distributed system: we're hard to debug, our interdependencies are a bit confusing, and there are a bunch of places where a small amount of added latency can cause Byzantine failure modes.
So the system gets slow. And all that intrinsic motivation…?
Technically we have autonomy, but small things are more complicated than they need to be so it's hard to feel like you can do anything.
There's not a lot of opportunities for building mastery in anything other than pushing forward through friction and dealing with ambiguity. Which are skills worth cultivating, but they probably aren't *all* you want to do.
And it's hard to believe in a purpose when you're not seeing any progress towards anywhere.
I think, for all that we do it, we know in our hearts that shiny-driven development is bad for us.
After years in this industry I've noticed something that every site reliability engineer writes at performance review time. Maybe this will resonate with non-SREs too, I don't know:
"I will focus more and do fewer things at once."
I'm not mocking. I write this too! It's because there is infinite work. There are always ten things that we can see that we could start to fix right this minute. And a lot of us are people who have trouble not fixing things that we see that are broken. So we end up with all of those open processes and without a sense of achievement.
We aren't really any closer to our goal. Jumping on problems made it feel like we were going faster, but we just made everything slower.
So what do we do instead?
We choose work deliberately. We aim at our goals and we ignore every. single. other. thing.
Even broken things.
One of my favourite quotes of the last few years...
So how do we know which things to do?
We ask why we would be doing them.
People used to use this model called the Five Whys method for debugging. It's been a bit debunked recently in tech because complex systems are complex and five whys is pretty linear, but the idea is that you don't stop at the first reason someone gives for why something broke, you keep asking why until you understand the underlying systems that caused the whole thing.
But even if it's not as useful for debugging any more, I like it when we're deciding what to work on.
Like, say someone wants to upgrade some piece of infrastructure to the newest version. You might ask "yeah, but why?"
Oh, it has high availability features. If we use the new version, we can seamlessly fail over from one machine to another.
Ok, why do we want that.
Oh, it has high availability features. If we use the new version, we can seamlessly fail over from one machine to another.
...ok, why do we want that.
We want to be able to shut down down a server without a bunch of manual effort.
Nice. There's a lot of reasons you might want that. What's the problem you're trying to solve?
We want to be able to do rolling restarts of the fleet.
Oh yeah? Why do we want those?
Because we want to have automated fleet management… and that’s a goal we chose.
I mean, that's five whys and sometimes we'll need to keep going for many more. But the point is, you'll either get to something on your civ tech tree that you decided to solve or you won't. If you get to a place you decided you were going, you can stop and say yes, we should do this thing!
But then… what if there's a thing that you didn't choose to do, and now you see it and you just really want to do it now?
And the answer is: yeah ok but don't.
This is something it took me a long time to learn: just because we can fix something, just because it would be helpful, doesn't mean we should do it any time soon. We can leave some things broken. We can nod to them every day as we walk past and see them still there. “Yup, you are still broken. Yup, that will continue to bug me. But I'm going to resist.”
Starting to fix the thing will feel faster. It feels more productive. But long term, having a bunch of things started just slows us down. We'll get to it when we get to it.
Ok, look, there are exceptions. But one of four things is true:
Maybe this is very small and a quick win and it'll block people more if you don't do it. In that case, sure, go for it. Fix broken documentation as you come across it. Clear traps out of your path. If it's a few hours work and it'll make everyone's life better, do it.
Or maybe it's a crisis, in which case, well that's not ideal, but you have to change your goals. If your database was at 90% yesterday and it's at 91% today and it's looking like it'll be 92% tomorrow, this is not the time for principled objections about how this wasn't in your goals. Go prevent the impending doom. But explicitly add it to your goals as a higher priority item than whatever you were doing next.
Or maybe it's not that important but it's just something that would be fun to do. In that case save it until a time that's designated for fun. It's good to have some time to let people play with ideas and fix local low-priority small stuff that bugs them.
I love hack weeks for this. We just had a hack week and I wrote a bunch of node.js and made a serverless app to solve a work thing that was bugging me. It was not even slightly the highest priority thing I should be working on, but I learned a lot and it was really fun. Some people do 20% time, which I think is also pretty cool. It just needs to be finite, dedicated time.
But a lot of times it's not any of those, it's this 4th state: it's something that's genuinely important and if you do it you'll make things better. But it isn't more important than the thing you're already doing. And in that case, ignore it. It's not yours. Well, write it down if you like, dump your implementation ideas in a jira if you have to. But make it clear that it's open for other people if they have time before then, and that they can ignore your ideas if they want.
This is so hard.
But otherwise, if you do the first 5% of it, you're implicitly claiming the project.
I've heard this called licking the cookie or licking the project, which I think is wonderfully disgusting.
You're not going to eat the thing immediately, but you want to make sure nobody else will take it either.
As individuals, it's really tempting to claim problems as our own. Partly because impact, partly because it feels good to solve things, but also because we care about how the thing gets done and we're worried we won't like how other people might do it.
But if you see a project you want to do and don't have time to do it right now, you have to leave it there. As so many inspirational posters in weird gift shops have told us…
if you love the project, you have to set it free. Even if you know in your heart you were meant to be together! Even if you would do this project best of all. Sometimes you have to let other people do things that you would do better.
One of my favourite metaphors of all time is Molly Graham's “Give away your Legos” talk.
Molly compares scaling an organisation to building an awesome lego project, and having to let other people join in and take over part of it. She says it's natural to have some anxiety that the new person will build the lego tower wrong, or that they'll take over and there won't be any lego left for you. But she says that's the only way to move on to building bigger and better things.
This goes double if you're a busy senior person.
In Lara Hogan's wonderful blog post, Manager Energy Drain, she says the best gift a manager can give their reports is a messy, unscoped project with a bit of a safety net. I think this applies to non-managers too. Don't start or scope out projects for people. If you do the first 5% you’ve implicitly claimed it. But if there's a project that needs to be done, do look around for someone who would learn a lot by having to figure it out.
And tell them that you'll help any way they need. You'll be their safety net.
Senior people can become the biggest bottlenecks in an organisation. Supporting someone else through doing a thing you could do much faster is another thing that will really slow you down in the short term. But longer term, you have another person who can go fast, and you'll go faster.
Whatever you decide, write it down.
Write down the stuff you're doing, the stuff other people are doing, and the stuff nobody is doing. Do include the stuff you decided not to do as well as the stuff you are doing, so you remove ambiguity. People don't have to live in the world where you may or may not have thought of the thing.
That's our second thing! Making decisions and writing them down! It definitely takes longer than not making decisions and not writing things down, but you will make up the time!
Look, here's an analogy.
Suppose you saw a system designed like this: There are a bunch of data sources. I'm drawing two on the slide, but imagine there are a bunch more.
And there are lots of servers that need information from the databases all the time. And the servers all need the same information, so they continuously hit the databases asking for the data. So the databases are under a lot of load. You might say "hey, maybe add a cache there?", and I would say yes but wait, there's more!
Because also the network they're crossing to get the information is super unreliable and there's very little validation or error checking. And if you're thinking "This design has some issues", I assure you that there's more!
Because when the servers get the data, they all need to process it using some sort of translation function, even though they're all getting the same raw data. BUT…
…each of them have different versions of the translation function and most of them are out of date. And also some of the service owners rolled their own code to do what they think the function should do. And also they're wrong.
And if any of the servers ever can't get to one of the databases, sometimes it asks one of the other servers for the data, and that server gives it a version of the data that has maybe already been run though the translation function, which again is probably wrong.
Oh yeah, and we'd like all of these servers to come up with the same results and interact smoothly and work well together. Would you approve this design document? I HOPE NOT! THIS IS A TERRIBLE DESIGN.
But somehow we’re ok with it when the same system is made out of humans. We're ok with it when everyone has to talk to the humans who have information we need, gather raw data and then figure out what they think is happening and share information based on their own mental models -- most of which are different from each other and some of which are just completely nonsense!
And then we're surprised when we don't all understand the same thing and projects don't work out!
What if we made the decision once! And wrote it down!
With the best will in the world, we all remember things differently.
We have a one hour meeting, reach a decision, and then everyone leaves with slightly different information in their fragile, lossy human brains.
I get stressed out even thinking about this, like in the same way I would if you told me you store your data in one place and you've never tested your backups.
I think people are scared of writing things down.
(Image: The Magna Carta is a charter of rights agreed to by a bunch of people in 1215. None of them stood behind the commitments from the first draft and the charter was later annulled but THAT IS NOT THE POINT. Later they made another version and that stuck. It’s in the British Library beside Euston Station, so drop in if you’re ever early for the Caledonian Sleeper.)
I think they're scared of being wrong! But it's better to be wrong than vague.
If you think you and someone else have a different mental model of how something will work, the most useful thing you can do is write down your understanding and show it to them all "hey, here's what I think is true. Does everyone else think this is true?"
Because then if they disagree, they'll tell you. If you're wrong, engineers will definitely tell you, I promise you that engineers will tell you. But if you handwave things, everyone will superimpose their own reality onto it and assume you agree with them.
Wrong gets corrected. Handwavy lasts until you try to integrate your system with someone else's system and discover that you meant different things and someone just wasted a quarter.
It takes time to disagree, but it takes longer to not disagree and end up going down the wrong path.
The more you spell out your understanding, the more people will notice how it diverges from theirs and let you know. So be really opinionated and really unambiguous. Watch out for words that can be interpreted in different ways.
At Lead Developer New York this year, Rod Begbie talked about "blur words", things that imply different things to different people. His example was "end of day". If I tell you I'll finish a document by end of day today, do you expect to see it at 5pm or midnight or like 8:59 tomorrow morning? I think the word "should" is like that. If I say "We should support multiple languages". Does that mean we will, or does that mean we won't but we'll feel bad about it?
Lots of the time there will be multiple good paths but the decision doesn't really belong to anyone, and so everyone is kind of skirting around it.
If there are decisions that nobody owns, have a go at making them. Even if you don't really care about whether you choose path A or path B, write down that both paths exist and just arbitrarily choose one of them. If someone really feels strongly that path A won't work and it needs to be path B, well at least you have more information, and you've forced them into making the decision. And then you update the document to say that you're going with path B instead. Decision made. Ship it.
But that only works if it's ok to be wrong.
It needs to be safe to change your mind. This is one thing you really need in your culture. It has to be ok to be wrong. You have to be able to course correct without penalty.
Otherwise nobody makes decisions or writes anything down and everyone gets wedged in a whole fog of procrastination and vaguely written design documents in the passive voice (and the passive aggressive voice which is much worse) and nothing ever gets finished.
It needs to be safe. If people don't feel safe, they won't take risks, they won't admit when they're stuck, they won't admit when they're not going to do something.
It takes time to be patient with people! It's slower! But then later you have a safe, blameless, awesome team culture and everyone does better work.
So be kind. Btw, being kind is different from being nice.
Nice is about your externally visible demeanor. It's about being polite and smiling at people, making peace and being agreeable. Nice is great! Do be nice when you can. But kind is more than that. Kind is more active.
Kind is about being invested in other people, figuring out how to help them, meeting them where they are.
Nice says "good job in the meeting." Kind says "your answer to the question was a bit rambly and you missed the opportunity to convince the team about your idea. But it is a good idea so practice your elevator pitch."
Nice brings in cake, and we love them for it. Keep bringing in cake! But Kind goes to your manager and says "hey this person is doing great work, consider them for the next lead role", and you never know it happened.
I mean, be both. Be nice as much as possible! Meetings and work and everything go better when people are trying to be agreeable and friendly with each other. But go further than that. Actively care about people and get invested in making them better.
People do much better work when they feel safe, and that's something every individual can influence.
But especially senior people.
If you're a senior person, you have so much power to change the level of safety in your team and organisation. All of the stuff I said so far goes double for senior people: un-lick more projects, make even more decisions, change your mind and course correct even more publicly.
But there’s even more than that. Ask questions so it's safe to ask questions. Admit you don't know everything and let people see you learning. Make it normal to continuously learn new things.
And say when things are good. Call out good work in meetings. Explicitly approve and endorse design documents. Engineers have a terrible habit of reviewing things by pointing out the things they dislike. Don't just point out the nits and move on. Do a high level comment where you say that overall you approve, if you do.
And be reassuring when people break things and tell them the story of times you've broken stuff. (If you claim you're senior and have never broken things, I don't believe you.)
Notice when people are blocked because they're procrastinating or because projects are too hard. Help them get unblocked. We’ll talk more in a minute about how to do that.
Make everyone feel safe and supported and you make them do better work.
You don't have to be a manager to nudge your colleague in the right direction.
I mean, this is for everyone, not just for senior people, but I think the ability to make other people better should be a requirement for being a senior person.
Senior people should have high generativity. And you might say "but wait, helping other people takes time! I am a busy senior person who needs to get projects done. I can't afford time for helping people". It's a wrong optimisation. If you make everyone better, you'll get where you're going faster and better.
Again, productivity is what you do yourself, generativity is how much better you make the team.
In the article I mentioned before Jessica Kerr talked about the systems interrelationships between people and how they end with a group that's greater than the sum of its parts.
She points out that it’s possible to be quite productive personally and have negative generativity, holding the rest of the team back.
And I'm not going to lie to you: our industry still is not great at rewarding generativity. I'm definitely not saying to spend all of your time on helping other people. Do your own work too and make sure your achievements are well documented. But if you can make a culture where people helps each other be more successful, you'll find it easier to get things done, you'll spend less time being blocked and you'll do better work and have more achievements. And have a better time, I think. It's really not a zero sum game.
People working together can aim at bigger goals. And they can reach them. They can finish things.
And finishing things is the best! You close a process. You make everything a little less chaotic. And then you get to do some of the shiny things you carefully left unclaimed before.
But completely finishing things is hard. Somehow, the last 10% of a project often takes as long as the first 90%. And when things get blocked, that's when we're likely to see some other shinier project and follow that trail instead. It's worth being really deliberate about closing things down.
I want to draw a distinction between things being finished and things being completely, perfectly completed.
Sometimes priorities change, or people leave the company, or the problem is discovered to be genuinely too hard, or it's become so low priority that it will never happen. Sometimes you have to admit a project isn’t going to happen.
It's very hard to throw away work. I mean, there's sunk cost fallacy, but it's also just hard to tell yourself you're not going to use something you spent a ton of time on. But it's better to be decisive, release the lock and tell people you're as done as you're going to be. If you fail fast and write a retrospective, that's a really good way to end a project. And abandoned work often comes in handy another time. You'll have learned something.
But you can only do that if it's safe. You need to have the sort of culture that rewards people publicly walking away from doing the wrong thing. If you're somewhere where it's not safe to cut your losses and admit defeat, projects will stay half-open forever.
But lots of the time, the project is not dead, it's just… stopped. It's just in this weird dormant state where you're not sure why it's not moving along.
To understand how to get things finished, I think it helps to look at why things get wedged.
Like, in theory projects are wedged because there's too much work and there aren't enough people.
But more likely, not everyone is busy, or at least not busy with important things -- they may be working on some local maximum project. More likely you’re blocked on specific people.
There are often a small set of people who are single points of failure and if they're not available the thing doesn't get done. Unclaiming projects and supporting other people gets us a lot of the way to greatness here.
But there are other reasons things get stay not finished. A lot of the time it comes down to procrastination.
And I don't at all mean that pejoratively. I've heard procrastination described as "slow terror" but I think is even more accurately called "fear of pain". We don't procrastinate because we're lazy! We do it because we're asking our brains to do something they just don't have enough space for, and that manifests as pain. And we want to do *something* but not the thing that we'll find painful, so we do small things that do fit there instead. Like doing lots of small tickets to avoid confronting the big design decision.
You can help procrastinating people, including if they're you.
One way is to add accountability (either for yourself or for other people) by setting deadlines, making commitments, and generally making it scarier to not do the thing than do the thing. This is the only way I get anything done: I sign my future self up to deliver something on a specific date, and then I have to go figure out how to get it done. Artificial deadlines focus the mind!
But there's a kinder way too and that's...
...to respect the fact that you're asking your brain to do something hard.
If you're trying to schedule a 5GB process in a brain with 500MB available ram, feeling bad about it won't make a difference. You need to increase the available ram, or break up the process.
We can help ourselves have more ram by doing whatever our biology needs. For me, eight hours sleep makes difficult things really easy. I resent the hell out of that because I like staying up late but it's highly correlated. Scheduling time in calendar early in the day helps me too -- I can work through it before the day's decision fatigue catches up with me. Different people's brains need different things. You need to be kind to yourself too, not just kind to other people..
Giving yourself more ram might not be enough. Maybe the project is just too difficult. Maybe it's more than you will ever be able to handle. Sometimes even the step of thinking about how to break up a project is too hard. Like the big wardrobe cleanup project earlier. It becomes easier to just not think about it.
The best solution I've found is to work as a team. A group of two people at a whiteboard can solve more than two people separately. There's that generativity thing again. There's something really powerful about people realising that other people also find things difficult. In a generative team, we feel safe getting vulnerable and honest with each other about what we don't understand or don't feel comfortable doing.
Being a team also means we get to bring a bunch of different skills to a project, and we can learn from each other. Because we all find different things difficult.
For example, I think that one of the ways a project often stops that we've done all of the well-understood parts of the project, by which I mean the technology parts, and all that's left is the messy human parts of the project. And not all of us find humaning easy, but we can learn from each other.
In his great presentation at Velocity last year Michael Bernstein talked about the phenomenon where someone writes a bunch of code, makes a good thing that people are likely to want, and then… stops. He said it was like a farmer doing a ton of work to grow food… planting, watering, weeding and then, when the food is ready, just stopping. You need to harvest the crop and take it to people. You have to convince the humans they want it.
Building it isn't enough. There's more work to do. But when we spec out a project to do a migration or build some new internal tool, we're often only creating jira tickets for the part of the work that is creating and deploying the software.
We need to track the non-technical work, the marketing, the adoption, the documentation, the cleanup. All of the talking.
And we need to consider the project not complete until it's all done.
A previous team I was in said they celebrated landings, not launches. You don't get to celebrate until users are actually happily using your system. If you're doing a migration, you celebrate that the old thing got turned off, not that the new thing got launched.
Some teams have a "definition of done". If it's not monitored, it's not done. If it's not documented, it's not done. It's worth getting a definition of done in place for your project that includes all of the talking that will need to happen.
I think a lot of the time, we have a bunch of very well scoped projects, but a lot of stuff falls into the cracks between them and there's work that's not anyone's job. That's why it's good to be clear on exactly what problem you're all trying to solve. If it's your job to solve the problem, all of the small stuff that doesn't fit into any specific project is also your job!
That way you won't just see the projects, you'll also see the negative space between them. And again, that work is very likely to involve talking.
As my friend Mark says…
People are great at recognising meetings that could have been emails, but they’re awful at recognising long email threads that could have been resolved in a 15 minute meeting.
I think this is really true. I've seen this a lot, where the thing that was missing was one person (often a TPM because TPMs are geniuses) wading into a situation all "ok, look, tell me what's done and what's not done?" and asking questions until people are moving again.
That's a skill anyone can learn, by the way. I always see these lists going around like "Top ten technologies we're hiring for in 2019" or whatever? Yeah, machine learning and cloud computing and etc are great, but I'm calling it: the #1 technology skill we should be hiring for is the ability to talk to other humans.
Because talking to other humans is the only way things get finished.
Ok, project done! Problem solved! If you have achieved the goal or the railroad or whatever you set out to achieve, it's important to take the time to celebrate it. It's not easy to finish things! You've made progress on your civ tech tree, and you've closed a long running process. You made it to the shining city on the hill!
Take the time. Feel good about it! Have an ice cream party! But then what. What's next?
Then you need to choose the next thing to do, which means another point on the Civ tech tree, another round of looking at projects and asking "yeah but why" and deciding what you're trying to achieve.
But the end of a project is also a good time to make sure you're still going somewhere you want, aiming at some sort of goals you have. It's time to ask why for yourself as well. Where are you going? What do you want?
It’s a good time to look at the job you're in right now, and go through the same yeah but why exercise.
Why are you in this job? Why does your job exist? (And should your job exist? I mean, if the answer is no, that's not ideal, but at least you figured it out before anyone else did?)
And why this career? Is this giving us things we want? Why are we doing any of this?
Why are we building these skills? Is technology actually a terrible idea? Should we be helping the robots get so smart? Isn't there a greater purpose in life? Looking out into the infinity of the universe, does it really matter what we do?
No wait, step back. That was too many whys.
But it is worth asking yourself why you're in the role you're in.
Are you there because you're learning a lot? (That's my #1 driver most of the time.)
Or is it a stepping stone to a thing you want? How will you know when you're ready to go get that thing?
Or are you there because it pays the bills and lets you pursue some other dream you have? That's extremely wonderful! Go kick ass at that other dream!
Or, to be clear, if your reason is that you're choosing to be somewhere you don't actually like because they're raining gold on you, then that's a perfectly valid reason.
The key is that you know why, and that you choose. The key is that it's deliberate. Because then you can start thinking about where you want to be.
I'm really not saying that you need to figure it all out. I once read a book that was like "stop here and write down your life goals and don't continue until you've done that" and it was like 15 years and four apartments ago and that's as far as I've gotten. True story.
But 3 years? 5 years? What do you want to still be doing? What do you want to be different? What would you choose if you could magically get there, and didn't have to start from where you were? Is there someone doing a job that you wish you could do, but you don't have the skills yet? Describe what it would be like if you were doing that job.
Choose the shining city on the hill and only then start thinking about how to get there from where you are, what projects are in the tech tree pointing to there, what stuff is in the cracks between the projects. It turns out that everything in our industry is a learnable skill. Every single thing. Anything you need to know, you can learn.
But the deal is that once you've chosen, once you're going somewhere, that's not the end.
We're never actually at the end. Maybe we have built a railroad, but now we get to go aim at something else and go around again, taking the time to deliberately choose what we work on, write things down, make a safe supportive environment for other people, and finish things and put them away.
It's all continuous after all :-)
The tech will change. The jobs will change. Today's hot technology will become tomorrow's legacy tech. At some point, doing the same thing today as we did yesterday will stop working.
The skills that will never go away are talking to other humans, learning and deliberately choosing a destination.
So continuously give yourself space to think where you might like to be.
Keep asking why and keep being existential. And that's all I have. Thank you.
Accelerate: State of DevOps, by Dr Nicole Forsden, Jez Humble and Gene Kim.
The Origins of Opera and the Future of Programming, by Jessica Kerr
A Theory of Human Motivation, A. H. Maslow
Drive: the Surprising Truth About What Motivates Us, by Daniel Pink is the origin of “Autonomy, Mastery and Purpose”
Give Away Your Legos and Other Commandments for Scaling Startups, an interview with Molly Graham
Manager Energy Drain, by Lara Hogan
Microspeak: Cookie Licking, by Raymond Chen (this isn’t where I saw the expression, but it’s the earliest reference I can find online)
Video for Being Right Is Only Half the Battle, Rod Begbie, which is the origin of “blur words”
Tweet for “Do less, well”, by @colmmacc
Tweet for “the 6 week email thread that could have been a 15 minute meeting”, by @utterlymundane
Video for “Why Marketing Matters”, by Michael R Bernstein
The Caledonian Sleeper is a wonderful train.
You can play Civ online at https://playclassic.games/games/4x-dos-games-online/play-sid-meiers-civilization-online/play/ (I’m sorry.)
Thank you so much to Christina Schulman, Dina Betser, Julia Schmidt, Joel Votaw, Kate Mahoney, Laura Nolan, Tiarnan de Burca, Andrew Lawrence, John Turner, Robert Konigsberg, Suse Goericke, Alex Hidalgo and Niall Sheridan who gave me comments on some of the many, many early drafts of this talk. To quote the feedback from an early practice run: “uh …how did you feel it went?”. See, nice would have been to say it was good; kind was to be clear it still needed some work and give a ton of quality suggestions. Thanks, friends <3