Story Points. A Friend Or Foe?

In other words, we suck at estimating.

But on the other hand, what should you do as a horrible estimator when the Scrum guide (till 2020 release) says that estimates should be provided by people that will be doing the work, but at the same time, it doesn’t tell us how we should provide estimates? And what did Agile enthusiasts do after reading the scrum guide back in 2008 when they could not relate to any step-by-step practical instructions?

Yes, they started estimating Product Backlog Items (PBIs) in time, and yes, they did wrong. Mostly because of:

1. Again, we suck at estimating.
2. Estimates are treated as a commitment by the stakeholders.
3. The outcome from the first two: estimations took a while or were overestimated.

The story point is an abstraction that solved these problems. It had no units, so it can’t be related to any commitment. If you asked the right questions (for example ‘How is this compared to that thing we did last time?’) it could save a significant amount of time on the estimation process since you just compare things one to another. And if you fail with the estimation, it’s fine as long as it’s consistent. Here is why. If you say that a particular user story is three times more complicated than the reference one, but in reality, it’s only two times ‘bigger’, it’s still OK if you estimate all similar stories that way. Just keep it consistent.

Where did story points come from?

Story Points concept was the Extreme Programming (XP) idea, which was adopted by Agile practitioners. As Ron Jeffries describes, the original “Points” for estimating ‘Story’ were based on ‘Ideal Days’ to finish a specific PBI that might have been translated to the real days by multiplying by the “load factor” to convert it to the actual implementation time.

For example, a story estimated as three ‘Ideal Days’ multiplied by the load factor of three gave nine days estimation.

But the stakeholders were confused with three ideal days estimation versus nine non-ideal days, so Ron’s team started calling ideal days ‘Points’ or ‘Story Points.

And that’s how the Story Point concept appeared – time estimation, that evolved into something much more prominent these days.

What is a Story Point?

A story point is an abstract measure of effort required to implement a user story. In simple terms, it is a number that tells the team about the difficulty level of the story. The difficulty could be related to complexity, risks, and efforts involved. Let’s briefly analyse each measure.

Effort or volume. How much is there to do?

• What amount of work is there to do?
• Is it big? Is it small? Is it average?
• How much effort will it take to complete the job?

Complexity. How difficult is the work?

• How complex is the work compared to other stories of the same volume?
• Is it easy or hard to do?
• What are the dependencies and circumstances?
• Are there any uncommon practices involved?

Risks. What are the unknowns?

• What do we know that we don’t know?
• What is our experience with similar stories?
• What are we not certain of?
• Does it challenge our skills?

Story point estimation is typically performed at the Product Backlog Refinement Sessions. Product Backlog is evaluated by the team responsible for the actual development and testing work.

Benefits of using Story Points

Jeff Sutherland, in his article, mentioned that ‘Story Points are therefore faster, better, and cheaper than hours and the highest performing teams completely abandon any hourly estimation as they view it as waste that just slows them down.’

Let’s review all the benefits Story Points bring.

Estimating in Story Points is fast. Teams who estimate in days tend to take the discussion a few levels deep. If you estimate in days, the fear of commitment plays a role here.
As said by Jeff Sutherland, one of the CMMI level 5 company determined that story point estimation cuts estimation time by 80%, allowing teams to do other productive project activities.

It’s Relative. Mike Cohn best described it in Agile Estimating and Planning ‘Because the estimate for each feature is made relative to the estimates for other features, it does not matter if our estimates are correct, a little incorrect, or a lot incorrect. What matters is that they are consistent.’ And Indeed, ‘there is credible evidence that humans are good in relative estimation compared to absolute’. (Lederer and Prasad 1998).

Story Points remain the same for everyone. It doesn’t matter if a Senior or Junior developer will deliver the story. What matters is that the story is X times bigger than the reference one.

Story Points allays fear of commitment. When estimating using hours, you make a precise time commitment. Estimating in Story Points prevents giving an exact commitment. Nobody knows precisely how many hours you are appointing to a specific Item.

Eliminates the need for frequent re-estimation. When planning with Story Points and velocity, we essentially take those points as size-based estimates. As mentioned before, we don’t expect the size of the story to tell us the time needed to complete it. If estimates are in days, the team finds itself in the position of needing to frequently re-estimate future stories based on current knowledge. When teams talk about “days of work” implies a level of specificity that isn’t real. We avoid this problem when using Story Points. The fact that a team learned to work twice as fast gets reflected in their velocity. As long as the story sizes are consistent (read the previous points) between stories, no re-estimation is necessary for planning to occur.

Story Points help drive cross-functional behaviour. Mike Cohn explains this very well in his book ‘Agile Estimation and Planning.’. Agile teams include people from different disciplines like programmers, analysts, testers, designers, or product owners. Agile Projects will benefit when each team member thinks about a built feature first and then being a specialist contributor. As such estimating in story points will help teams learn to work cross-functionally.

Since Story Point is a single number, all the teammates need to think about what they estimate from a big picture and as a complete feature. They need to shun their departmental mindset of how much programmer or tester or UI person would take and try to add the estimates. But at the end of the day, the scores are made up, and they’re numbers. It’s the conversation and collaboration during the session that really matters, brings value and helps to gain a common understanding of stories. The team behaviour becomes more prominent over individuals as conversations help team building, team members constructively criticize each other, and debate.

Weaknesses

Story Points are not intuitive. My job provides me with the possibility to communicate with lots of Agile Poker users, Scrum Masters, Product Owners, and Tech leads. I regularly hear that they very often don’t understand the Story Points concept. Not only the person representing the concept has to repeat basic principles several times, but also the practice shows that the starting 3-5-10-… Story Point sizing will be full of questions and arguments:

Should we go for another round?

1. Why don’t we just take the bigger number?
2. I will do this in 3 SP, why are we putting 5 SP?
3. Why can’t we use the average of 3 and 5 SP?
4. What is the difference between 1, 2, and 3 SP?
5. What is the lowest SP value, 0.5 or 1?

Moreover, have you ever seen somebody saying ‘it feels more like a 3 to me’ while all other members estimate as 2? I experienced such deadends in reaching a consensus. Eventually, it slows down the process and neglects the ‘Fast’ benefit. While, for example, the T-Shirt size scale is much easier in adoption. As Linda Cook mentioned: ‘Scrum Masters often guide the team to the desired answer rather than let the team arrive at their answer based on conversation and understanding of the work. Even worse, I have witnessed several Scrum Masters declaring the story point value once the story is read.’

Not truly relative / Translating Story Points to hours. Do you remember the story of Story Points? It evolved from time estimates, and even today, it is still mixed with time estimates. Teams keep converting time to Story Points and back since it’s natural and easy to understand. But why is that? Imagine a team estimating numerous PBIs, one by one. The group is supposed to compare each item to a baseline item (typically a ‘1’). It’s exactly where the main problem lies. The participants have to figure out how many times the item is bigger. The human brain tries to simplify the task by creating an image of each size on the scale and unconsciously translates it to time intervals (e.g. 2–3 days for size ‘2’).

The whole Story Point concept turns into absolute estimations without you noticing. It’s funny how time estimates evolved in relative Story Points to come back to the absolute estimates. However, in this case, you stop benefiting from the speed of relative estimation, so most of the benefits from the previous chapter don’t matter anymore.

Adjusting Story Point estimate with a specific developer to work on it in mind. Although this is one of the SP benefits, we see that our clients keep doing the following: a PBI might be 3 Story Points for a senior developer, but 8 Story Points for a junior developer, so the developer is selected first and then the SP value is set. It’s so easy to go with 3 SP in this case (and numerous Agile Poker clients do so!), but it’s wrong since SP estimate is a team commitment. What will happen if the junior dev will have to pick up the story?

Sizing unfinished issues again. The Story Points concept doesn’t provide a solution of what to do with stories that are moved to the next Sprint. Many teams re-estimate the story and put remaining estimates, while it is unnecessary to re-estimate at all. Think of the reference issue that has been re-estimated. This will endanger future Plannings. As a result of Sprint Planning, the team will know all necessary tasks to complete the issue. The estimation of these tasks is in hours. So in the next Sprint, the team will know how much time is still necessary to complete the PBI. The fact that the PBI was not completed will reflect the velocity.

Avoid adjusting reference PBIs. As an outcome of the previous point, revised reference PBIs lead to re-evaluating the team’s Story Point values and could seriously damage planning.

Summing up Story Points is artificial. Summing up the relative value of Story Points is an arguable process. Eight 1-point tasks take much less time than one 8-point task! Point totals across sprints are not comparable because of this math problem. While most of us try to forget this problem silently, some frameworks went even further. The exceptionally popular Weighted Shortest Job First (WSJF) prioritization framework suggests calculating the score based on 4 Story Points criteria. You sum and divide relative values to get another relative value for comparison purposes. It frightens me a lot. But still, it works.

Story Points vs Bugs and Tasks

From guides, you know that only User Stories should be sized. From real life, you know that teams estimate all possible issue types. Full stop.