25 Agile metrics and measurements - Measure one damn thing!

You and I know it: There are loads of agile key performance indicators, metrics and measurements. But you always have to keep one thing in mind:

“Tell me how you measure me, and I’ll tell you how I’ll behave.”

Dr. Eli Goldratt

Therefore, one question has to be asked: Are all of these metrics relevant? \n\nAnd then there is this other citation:

“Simplicity is the soul of efficiency.”

Austin Freeman

So the next question arises: If we would want to measure agile as simply as possible, how would we measure it? What agile key performance indicators, metrics and measurements matter most? Or if we would measure only one damn thing, what would it be?

Don’t get me wrong: Of course you can have more complicated models for agile key performance indicators or metrics, such as the Agilometer. But I was fascinated by the goal to measure one thing only. Therefore, this article investigates this goal. 

The above agile KPIs serve as the basis for this text.

A note in advance: A simple, pragmatic but very helpful agile metric is the one shown in the video here. Agustina even shares the team data publicly from the whole of last year:

Play

Do we even want agile key performance indicators, metrics and measurements?

Before we go deeper, one thing that I regularly hear or read on LinkedIn when I talk to agile coaches, scrum masters or scaled agile frameworks consultants: Do you even want to measure agile?

“There are lies, damned lies, and then there are statistics.”

Mark Twain

Mark Twain is a little dramatic about it. But he has a point. A point that Albert Einstein put in a Nutshell.

“Not everything that can be counted counts.”

Albert Einstein

What is “KPIs” in Agile - Velocity, burndown charts, number of failed deployments? Are these agile metrics critical to agile success? I doubt it. 

But abandon all metrics? That would also be a mistake.

In today’s world, economical circumstances change fast. In good times, decision makers start thinking about an agile transformation because they have the resources and safety to do so. And in bad times? 

In bad times, senior executives will fall back into traditional thinking. They will move to old behavioral patterns: e.g., top down decisions that are not compatible with modern agile thinking.

If you would accept not having agile key performance indicators in bad times, you shouldn’t even start an agile transformation. Because at that point, any progress made within that transformation would be destroyed.

Therefore, the only chance that scaled agile frameworks survive difficult times is by beating the system with its own weapons: by providing metrics. Agile key performance indicators that help to manage in times of uncertainty.

And I am sure about it: there are agile metrics that matter. 

“In hindsight, I consider it one of my biggest mistakes to always categorically use metrics for agile transformation transformation.”

Marcus Raitner

Einstein also says it in his citation: There are things that count. Those are the ones we are looking for in this article.

Measuring Agility - What makes a good agile metric?

Let’s say the question you typically ask in your daily stand-up is: What did you accomplish today? So that’s how you “measure” progress in your team.

Good question, right? No, not right. This question pushes the team to show that they are diligent. The question puts pressure on the team to complete the “to-do list” so that they can proudly refer to the metric: Yes, I’ve been pretty busy in the last 24 hours!

But completing the “to-do list” is good or not? Well, it depends. Something else is much more important: achieving the team’s goal. This is typically - in the case of agile teams - delivering customer value or, in English, “delivering value”.

Agile KPIs - how to measure agility

A better question (or metric) in your daily stand-up would therefore be, “How have you helped your team or organization achieve our (sprint) goal in the last 24 hours?”

Change your question (or the metric), change the way people think and act: first effective, then efficient - to put it in the words of Peter Drucker accept.

Change the metric - change the way people think and act.

To put it in Einstein’s words: We have to find the one thing that can be counted - and that really counts.

What does really count in an agile transformation?

How agile metrics and measurements should be viewed

The goal of your agile transformation is definitely not an agile transformation. Why are you doing the agile transformation? Let’s use the “3-Why technique” to understand that:

So the core reason for your agile transformation: you don’t want to end up like Blockbuster, ignoring industry trends and customer needs, unopen to change and finally running out of money.

You want to end up like Netflix, who are constantly evolving their business model around their customers’ core needs. Just look at them Case Study Blockbuster vs. Netflix.

How can you translate that into measuring one thing in an agile transformation? Well that’s tough.

What companies do instead: measuring parameters because they are easy to track. Because tools out there throw them at you. The probability that these are the right metrics is pretty low.

Because for example, it is not about improving sprint velocity. That is a common mistake: monitoring and measuring effort, or efficiency. Instead, it is about solving customer needs. 

Agile KPIs & Measuring Agility - an important insight

Having laid down all of this, if we want to measure one thing in agile, we have to rate the validity of this metric against one thing.

We need to measure outcomes, not outputs. We need to measure our agile transformation by how much it helps us achieve our goal. We must not measure people by their time spent, but by their contribution to a common vision or goal - we must measure the value created for the customer!

If we want to measure by value, we have to deeply understand the customer’s needs.

For example, a railroad company has to understand that it is not in the railroad business. It has to understand that it is in the transportation business. Customers do not care if they are transported by train or by plane.

One other thing: Of course, it is key to not only measure but also reflect on agile metrics. Like for example about the Spotify Health Check ´ results (through a retrospective). 

You can do exactly that (also across teams if required) using our Health Check & Retrospective Tool. You can find more information about this at “How it Works”. You can also simply take a look at a Health Check Retrospective here. In this case, it is a retro regarding Scrum.

Note: This retrospective format asks for agreement with the given Health Check items on a scale.

• Planning: Backlog refinement in our team is efficient and effective.
• Customer orientation: The planning of our sprints is always based on achieving the greatest possible customer benefit in the given time.
• Agile education: Team members, Product Owner and Scrum Master share the same understanding of their respective roles in the team.
• Scrum Events: Lately, every Daily in our team has paid off.

The measurement of culture metrics (such as psychological safety) can therefore take place in agile retrospectives, where measures can also be derived immediately.

One alternative to this is one-to-one meetings between managers and employees. 

Even in this routine, regular short pulse surveys and reflections can help to collect meaningful culture metrics and at the same time implement a continuous improvement process.

Our Echometer tool can also help you with this. The following template contains some health questions that you can reflect on individually with your team members. You can answer them on a scale from 1 to 7. 

Try it out without logging in using the button:

One other thing about agile metrics that matter - timing

One other thought is important here. At best, metrics depend on the timing and/or at what stage you are in your agile transformation. 

Let’s imagine that you already know that transforming to scaled agile frameworks or other agile working practices is the right move for you (That is by the way the first thing that many companies forget to check).

In that case, at the beginning of your agile transformation your focus should be on one thing: The agile mindset of the leadership team.

Is the leadership team really ready to change? Does it understand the implications of a transformation? Is it ready to be the first team in the company to seriously introduce agile methods - including Kanban, agile retrospectives & continuous self-reflection?

In theory, an initial agile metric should therefore focus on the “willingness of the management or leadership team”. My colleague Jean provides in his article 7 tips on the role of leaders in agile transformations .

The next most important question to ask in your transformation: Do we have the right processes in place to understand and monitor our customers’ needs continuously? This is where the next agile metric could focus on.

But wait. That doesn’t help us achieve the goal of this text - only one Thing to measure.

No. It serves to inspire you for your agile transformation.

So let’s take a look at what typical agile metrics exist and to what extent they correlate with the most important outcome that your agile transformation is about: customer benefit.

The most important agile metrics - a ranking

You can find a table above with the agile metrics that matterwithin scaled agile frameworks and agile transformations. I ordered them by five areas, five goals that you generally want to reach through your agile transformation:

• Customer value: Are you meeting the customer’s needs?
• Predictability: Are you delivering on time with smooth processes?
• Productivity: Are you getting more and more done at the same time and with the same resources?
• Quality: Do you deliver a product free of bugs and other issues?
• Culture: Are the people in your organization satisfied, continuously learning and do they feel free to innovate, making sure you can maintain your pace?

Agile KPIs - How to Measure Agile Success

The table above also gives an indication of

• how easy it is to measure the metric (based on my personal experience)
• how much that metric is correlated to our core goal with a long term perspective: future customer value (based on my experience).

So which agile metric from the table above is the one?

Interesting to note is that there does not seem to be an agile metric that is easy to measure and provides the right kind of value. Damn it.

I would say that “simplicity of measurement” is not as important as “correlation to future customer benefit”. Therefore, the most valid metrics seem to be “Usage Index”, “Customer Satisfaction” or the “Net Promoter Score”.

By the way, you can start collecting 11 KPIs in your team today by carrying out the so-called Spotify Health Check with your team. Take a look at this video and the other videos mentioned to see how it works:

Play

Scaled agile Frameworks - the most important agile KPI is…?

It does not matter if you use the Scaled Agile Framework (SAFe®) to measure one thing, or a different framework to do your agile transformation.

Given these three agile key performance indicators, what correlates most with future customer value is probably customer satisfaction. Or, to be more specific, top-box customer satisfaction. You can find more on that topic following this link.

Customer satisfaction probably also has the highest correlation of all of these metrics with the ROI of your agile transformation. It shows you whether to pivot, persevere or perish.

Agile KPIs: Is a simple metric enough - or not?

But stop. By focussing only on customer satisfaction, how do you make sure you are staying ahead of your competitors long term ahead? How do you make it possible for innovative and disruptive ideas to grow and thrive in your company? What is ultimately your long-term goal…

Given these questions, I think we are getting back to the basics: to continuously provide value and to innovate around it, you need two other things: smooth agile processes and a great company culture.

Your company culture makes sure that employees feel psychologically safe , are open to fail, raise their voice and share their ideas. And your agile processes make sure you implement these ideas faster than your competitors.

**Agile, as a leader, aims for a people-first approach; it puts people above things.**Vikram Verma

The following graphic illustrates this in an easy way. If value is your long term goal, then our input is your working methods times culture.

Culture × Agile processes = Long-term customer benefit

I once read that “Agile transformation requires culture change, not process change.”

I disagree with that. It requires both.

Scaled agile framework - Measure one thing? Measure three things!

So if you only want to measure one thing in Scaled Agile Frameworks (SAFe®) or other frameworks, then it would be customer satisfaction. But honestly, I can’t recommend measuring just one thing - I’m sorry to disappoint you.

If you really want to keep it as simple as possible with your agile key performance indicators and metrics, I recommend that you measure a minimum of three things that will help you thrive long term:

• Measure customer value - through customer satisfaction. 
• Measure Corporate culture - by psychological safety as an indicator for learning and innovation. 
• Measure agile effectiveness - using the “planned-to-done-ratio” as an indicator of how well you are able to deliver incremental customer value.

After measuring, build. Then learn. And then iterate on your metrics…

That’s so easy, right? Nope. Are you serious about implementing agile frameworks throughout your department or company?

What is a good KPI in agile and can I do about it?

We are currently surveying dozens of experts - Release Train Engineers, Agile Coaches, Scaled Agile Framework Consultants - on KPIs and metrics in the context of Scaled Agile Frameworks and agile methods.

Based on those interviews, we developed Project Scagile: 7 webinars to help you avoid the 7 core mistakes in agile transformations to avoid. One of the webinars is on the topic of “Agile Metrics”.

By the way, if you are still looking for a suitable retro board, our article can help you with the topic: Comparing the 6 best retrospective boards