This episode explores the benefits for setting and agreeing to criteria before making decisions.

Podcast: Download

Subscribe: Apple Podcasts | Android

You can’t know if you’re ready to release unless you know what done means. When you use release criteria to know when a project is done, you have taken potentially hidden decisions and made them public and clear. Make your release criteria objective and measurable, so everyone on the project knows what they’re working towards. Use the criteria as you progress through the project and up to the final release. Then you can say ‘Release it!’ with pride. — Johanna Rothman

The quote above is from a fantastic white paper on the subject of release criteria. It’s an easy, accessible read and I recommend it if you’re looking to tighten up the way you make decisions for software product releases and projects in general.

The mindmap below outlines the points in this episode.

Benefits and Uses of Release Criteria mindmap picture podcast

Benefits and Uses of Release Criteria

A good example of establishing release criteria is the process I kicked off in the Fedora Project back in 2009.

Related Episodes

Credits

All songs licensed under Creative Commons

Transcript

This transcript was created automatically and may contain small errors.

Today, I want to talk about criteria. This came up recently in the context of a software release, but I think it can also apply really well to meetings and probably all other portions of your life as well. So I want to talk about criteria. Release criteria is kind of a common term that’s used in the software world. The idea behind release criteria is that you decide in advance what the criteria for releasing your product will be.

So typically you have maybe criteria around quality. So the software must meet a certain level of quality, meaning that maybe all of your tests have passed or a certain portion of tests must pass or No priority 1 bugs are open. Maybe you have some bugs that are open, but maybe bugs over a certain classification must be fixed before you can ship. It could be a calendar date. It could be, well, we’re shipping no matter what on this date, and that’s your criteria.

It can also apply to meetings. So maybe you’re in a meeting, maybe you’re having a specific meeting to make a particular decision. So deciding in advance, What’s important about the decision or how you will decide this particular issue in advance can also help drive the conversation. So if you set out at the top of the meeting to say, okay, we are going to decide how we will ship this product, but the decision is going to be made by one team only, then you could ask yourself, why are we even having this meeting if this decision is going to be made by this one person or this one team? On the other hand, if you’re deciding, well, this decision is going to be made by these 3 teams, and you have 5 teams at the meeting, then it’s really clear that maybe those other 2 teams are giving input, but they don’t have the final call on the decision being made.

So some other benefits around setting criteria is you create a common language. So people have a common understanding of what we’re talking about, why we’re talking about it, what matters and what doesn’t. It also kind of provides a common filter to see that particular topic through. So if you’re making a decision, as people are having the conversation and trying to get to this decision, it puts some framework, kind of a framework or some boundaries around what needs to be discussed and what doesn’t. In the context of a software release, say you’re having a discussion about bugs that block a release and whether a particular bug needs to be fixed or not.

If you’ve decided in advance what makes for a critical bug or a P1 or whatever your terminology is, if you’ve established in advance, you know, these or a term that we use a lot is a blocker bug. So this bug blocks the release. It must be fixed to unblock and ship the release. So what’s a blocker bug? And sometimes this is a fun one because sometimes people say, well, you know, I know one when I see it.

Except that the problem there is if you put 5 people in a room and say, well, do you think this is a blocker bug? You often can’t get to that immediate consensus because opinions or perspectives differ. So this might— this is a great situation where you could say, well, it’s a blocker bug if it impacts the install path. So if the product can’t install or If a key component crashes. And you can say, well, what’s a key component?

So you might want to list those. Well, if it’s one of, you know, maybe you have 10 different components that make up your piece of software, and the key components are these 3, then you’re saying in advance, these are the 3 critical components. And if one of these 3 critical components crashes, it must be fixed to release the product. Okay, now we’re really clear. So there’s a problem in one of the other components, it might crash.

It might look really, really bad, but you’ve already decided in advance it won’t stop shipping the release. So another benefit of discussing criteria in advance like this is you— is there’s kind of an expected outcome. So people kind of are clear and you’re, you know, you’ve established kind of this common language. You’re kind of thinking about things in the same way. And now we kind of have the same understanding as to what’s going to happen at the end and how we’re going to get there.

So what does this look like in real life? Like, what would be a real— I’ve kind of talked about this already, but kind of a real-life application of where you could use this? Again, a meeting. So how a decision will be made. Maybe starting with what decision needs to be made to begin with.

That’s interesting. I’ve seen so many meetings that I’ve had where it was like, okay, so we are here today to make a decision about X. And I’ve had some meetings completely stop right there. We are? We’re here to decide that?

Can we decide this? Oh, we can’t. Or I thought that was decided last week. Why are we even meeting today? Oh, well, that’s some good stuff to figure out in advance.

And I can’t count the number of times I’ve been to meetings where certain assumptions are made about things either being in place, being decided, or being clear.

The takeaway here is sometimes you can cut through all of that by just spending a few, a minute or 2 or less to make sure that it is clear. So again, back to my example of, you know, a meeting, Getting really clear, like, do we even know why we’re here today? And stating the obvious at the top can help big time. Software, I’ve kind of already talked about that. Again, knowing what will gate a release and what needs to be in it.

So what does this look like in real life? First of all, I highly recommend writing it down when you have these conversations. And some people hate having these conversations, but I think they’re a necessary evil. And that conversation being, So how are we going to make this decision, or how will we define what kind of bugs block a particular release? So writing it down is key, because once it’s written down, you can refer to it, you can come back to it, you can build consensus and agreement around what’s written down versus the, the oral tradition approach, which I have seen not work very well, if at all.

So once you write it down, put it in an easily accessible place. That’s, you know, maybe a a Google Doc, a webpage, whatever your company or your team works with, kind of a central place where they normally store information, that’s a great place to put it. Also, if you can give it a URL that’s easy to remember, so it’s easily retrievable. The worst place possible to store this information is email. I think it’s great once the decision has been made to send out an email saying, you know, maybe copy and pasting all the information into that email so that people can easily read it without having to go to this shared doc or webpage.

But don’t leave that as your central repository. Email is impossible to sort through and find later. In my experience, trying to find those emails later is just, it’s too frustrating. It takes too much time. So central webpage is easiest.

And you know, if someone’s deleted the email, whatever, having that URL, that webpage, where that information lives is key. And then the last thing I like to do in terms of executing on this idea is at the start of a meeting, spend 10 seconds, 15 seconds reminding people what the criteria is or pointing them to the page. So it could go as, it could be as simple as, today we’re here to discuss whether or not to ship product X. As everyone remembers or recalls, We decided what the criteria for shipping this product would be at our meeting 5 months ago. A link to that criteria is here.

And during this meeting, we will be discussing these topics and thinking of them through the filter of the criteria we set at this other meeting, which is on this page. Are there any questions about the criteria? Does anyone need a refresher before we dive into things? What that does not mean is that this is an opportunity to completely revisit the criteria and change it all. That’s a complete recipe for disaster and a meeting that you don’t want to have.

Now, I’m not saying it’s not worth revisiting the criteria or changing it, but I think it’s important to be intentional about changing the criteria. So if you’re sitting down to have a meeting to review bugs that should block the release, You’re not sitting down to have a meeting to revisit the criteria. So what I like to do in situations like that is just table that and say, we’re not here to discuss changing the criteria. This criteria has been serving us well. If there’s a glaring problem with it that we can discuss quickly, great, let’s do that.

If not, if we need to have a longer, deeper meeting to revisit the criteria, let’s set up a separate time to do that. And, you know, worst case, you can say, okay, we were here today to discuss bugs and, you know, triage these bugs and really reduce the bug count. If we don’t think that we can do that today without spending, without spending, you know, a good block of time reviewing the criteria, let’s reschedule that meeting and instead use this meeting to revisit the criteria. If we have any time left at the end, we’ll review bugs. If not, you know, we’ll have to have this bug meeting at a separate date.

I find that’s a good way to just kind of put some boundaries around it and break it down and then move things forward. So I’m curious what people think of this idea, this topic. If you have a situation where you’ve done this and if it’s worked well, I’d love for you to share it with me. If you have a situation that you’ve tried this in and it hasn’t worked, or you have a situation where you’d like to take an approach, but you’re not quite sure what to do, send me an email. I’d love to explore it with you, possibly cover it here on the podcast. Send that to REDACTED.