Thursday, November 3, 2011

Are We Done Yet? Or - Earned Value Calculations in Agile Projects

Don't worry, this is not one of these posts where someone attempts to cram Scrum burn-down charts into waterfall CPI/SPI to satisfy upper management who don't speak agile. This post is going to describe a metric that can be useful to project managers running software projects using agile methodologies. The metric is going to tell you how close to "done" you are in your project. Or more precisely: how close the value delivered by your project so far is to the total expected value of the project.

I have been using agile software development methodologies for a few years now. One thing always surprised me - while agile has effort tracking figured, described and codified as a set of useful practices, tracking earned value for projects is lacking. Up until recently, it was hard to find any mention of agile value tracking on the internet. These days, some authors write about it, but somehow the subject is not very popular in agile teams.

The Confusion

I have witnessed time and again that project teams tend to confuse effort spent (or estimated) with value delivered (or promised). What the popular metrics (like agile burn-down charts or traditional CPI/SPI values) do measure is effort, time and budget spent, and how they compare to the plan. And while very useful, effort metrics are not equivalent to value metrics. Not even close to equivalent to be exact. Why? Quite simple. Sometimes features that are very valuable are not very costly. Sometimes the opposite happens - you are working on something that requires a lot of work, but the return on investment is questionable. In a typical sofware project, a relation of a cost of a feature to its value is vary, very non-linear. Yet, the fallacy of equaling effort spent (or, in other words, sunken cost) to value delivered is very very common.

Why is knowing the value delivered useful? Well, it lets you decide whether it is worthwhile to continue the project or not. In fact, under normal circumstances, this metric should be the metric deciding the fate of your project. It is much more important than budget, time and effort overruns: you will want to continue a project that continues to bring you satisfactory amount of value regardless of whether it costs more than you expected or not. And you will want to cancel a worthless project even if it is cheap, because you can redict your people and resources to more promising undertakings.

Value Points

To measure value delivered by your project as you keep working on it is to add one more parameter to each of your user stories. The metric is called Value Point (VP). Value Point is vary much like a Story Point (used for effort estimation and tracking), in that it is relative. It tells you how the value of a story compares to the value of some other story. Which is useful, because it lets you estimate stories for value using not just concrete financial metrics, but also more fuzzy things (like gut feeling). 

Who assigns the Value Points? The ones responsible for this are project sponsors and any other party that have vested interest in the outcome of the project, can benefit from it, or can provide input for a value of particular story. How is the value points assignment done? the simplest way is to use an equivalent of Scrum release and iteration planning poker and calculate the average of VPs assigned by each participant of the planning meeting. A Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, 34, 55, 89...) is used to establish "legal" values. That is the methodology that is suggested by majority of texts on the subject.

However, other methods can also be used and are sometimes more adequate. For example, there is nothing sacred about using Fibonacci sequence here. You can use a continuum of values, fractional values, span many orders of magnitudes (this is a no-no for Story Point-based effort planning, but for value estimations, it is not uncommon to be able to tell with confidence that something is 200x more valuable then something else in the same project). Best of all, as long as the decision-making team is stable, you don't really need to reach any sort of a consensus and to pick an average Value Points number. Instead, you can just calculate a total out of everybody's assigned Value Points and use this as a final VP number for a story (in contrast to this, in effort planning, a consensus is absolutely required)

What Are Value Points Good For?

After you assigned value points to stories in your project, you are able to provide very useful metrics to the project management team
  • is the project still delivering as much value as it did when we first started it? 
  • If the value stream dropped, is the value being delivered high enough to justify continuing the project?
  • what is the percentage of "very valuable but easy to implement" features? To establish that, you pick stories that have high VP metrics and low Story Point metrics
  • what is the percentage of "not very valuable but hard to implement" features? If you have stories like that, you may want to consider dropping them (or maybe postponing until they become more valuable)
  • last but not least: how "done" are you in the project - meaning - what percentage of value points has been completed in the project. Upper management very often craves for this sort of a metric