Actionable metrics vs vanity metrics, with examples

A metric is actionable when it connects to a goal and the team can see what to change when it moves. Strong actionable metrics share five qualities:

Actionable does not mean complex. Conversion rate, cycle time or customer retention can all be actionable when they point to a decision.

Vanity metrics are not false. They are incomplete: they describe activity or scale without showing whether the work is producing the outcome. Compare the two for common situations:

The weaker metric usually counts volume. The stronger one adds a ratio, a segment or a link to the outcome, which is what turns a report into a decision.

In The Lean Startup, Ries suggests three tests for a useful report:

Start from the outcome, not from the data you happen to have:

1. 1

   Name the outcome. In plain words, such as 'more repeat purchases' or 'more predictable delivery'.

2. 2

   Find the behavior behind it. What do customers, users or the team have to do differently for the outcome to improve?

3. 3

   Pick a metric close to that behavior. Close enough that the team can see its own work move it.

4. 4

   Segment it. Break it down by channel, cohort, product, region or work type where that changes the decision.

5. 5

   Set a review rhythm. Decide who looks at it, and how often.

6. 6

   Define what triggers action. Agree what happens if it improves, drops or stays flat.

7. 7

   Add a balancing metric. Pair speed or volume with quality, cost or customer impact.

A number on its own rarely tells you what to do. Here is an illustrative path from a churn report to an action:

Skipping straight from the first row to a decision, say a win-back email campaign, would have treated a symptom.

The principle is the same everywhere: measure what can guide the next improvement.

Daniel Vacanti's Actionable Agile Metrics for Predictability argues that a few flow metrics forecast delivery better than estimates. They describe the system, not individual effort:

Velocity and story points can help one stable team plan, but they are not a productivity score and should not be compared across teams. People who expect a metric to be used against them learn to move the number instead of the outcome. A cumulative flow diagram shows several of these metrics on one chart.

A single metric invites tunnel vision. A balanced set usually covers four types:

A support team that tracks only tickets closed may rush. Adding reopen rate and satisfaction makes the picture honest. A delivery team that tracks only throughput may miss falling quality until customers notice.

Ask these questions of every new metric:

• A clear goal: It connects to an outcome the business has chosen.
• Influence: The team can move it through realistic action.
• Real value: It reflects customer, business or delivery value, not just effort.
• Segmentable: It can be broken down to reveal patterns.
• One definition: Two people would calculate it the same way.
• An owner and a rhythm: Someone reviews it on a schedule.
• A known decision: You know which choice it informs.
• Side effects considered: You have asked what gaming it would look like.
• A balancing metric: Something guards the other side of the trade-off.

Retire or rework a metric when you notice these:

1. 1

   Nobody can name the decision. The metric has no job.

2. 2

   It never changes priorities. It appears in every report and nothing happens.

3. 3

   Definitions get argued in every review. The number cannot be trusted.

4. 4

   The number improves while the outcome gets worse. People are optimizing the metric.

5. 5

   The dashboard grows and clarity falls. More numbers, fewer decisions.

6. 6

   It is used mainly to blame. People start hiding problems.

Send the numbers before the meeting and spend the meeting on decisions. Habits that help:

• Trends over snapshots: One data point misleads. A control chart separates real change from normal variation.
• Segment before concluding: Averages hide differences by channel, customer type or team.
• Speed with quality: Faster only counts if quality holds.
• Watch leading indicators: They move before the outcome and give earlier warning.
• Record decisions: Note the action taken and which metric will show whether it worked.
• Prune regularly: Remove metrics that no longer inform a decision.