Agile estimation with Monte Carlo simulation

By: Rich Stewart
Published Date: 25 March 2019

Much has been written in recent years about the importance of agile teams delivering value for their customers. The mantra is often “outcomes over output” and product owners in many organizations are rightfully starting to focus on ways to measure outcomes. While this shift represents a natural maturation of agile delivery in organizations, the importance of predictable delivery of features (i.e. output) remains an important goal for agile delivery teams.

Predictable delivery requires some method of estimation and many estimation techniques have been developed for teams using agile delivery practices. One of these techniques is Monte Carlo simulation. While this technique has been presented at conferences, made available as a spreadsheet tool, and incorporated into ALM tools, to this author’s knowledge, it has not been previously been made available as a free web-based tool. To address this gap, a freely available web-based tool has been developed that can be accessed on the Sevawise Games website.

The tool is straightforward to use and provides sufficient flexibility to fit the needs of most agile teams. It is primarily intended for Scrum teams that use Velocity, i.e., number of story points accepted in a Sprint, or Story Count, i.e., number of user stories accepted in a Sprint. The tool forecasts how many story points or user stories can be completed in a Program Increment (PI), i.e., a time-box consisting of multiple Sprints. For teams using Kanban or Scrumban, the tool can be easily adopted by measuring the Velocity or Story Count over past time-boxed periods (e.g. two weeks, a month, etc.).

As shown below, the user selects an historical data type of Velocity or Story Count and the number of sprints per PI. Historical data can be input from an Excel file (as shown below) or manually input into the datagrid. The number of sprints of historical data is manually input only if the user chooses to input the historical data manually.

 

Monte Carlo Simulation Input Form

 

The output data grid displays the level of confidence that a specific Velocity or Story Count will be achieved for the PI timebox. Note that the PI Velocity is not the same as Sprint Velocity as it represents a total number of story points likely to be completed in the PI time-box with some level of confidence. Similarly, the PI Story Count is the total number of user stories likely to be completed in a PI time-box with some level of confidence.

The histogram displays the distribution of PI story count or velocity based on the results of the Monte Carlo simulation. It provides a visual representation of the likelihood of specific ranges of results.

 

Monte Carlo simulation output results

 

The tool also enables the user to do an Excel export of all the input and output data displayed on the screen, except for the histogram.

To improve the usefulness of this tool, historical data should only be used that reflects the current maturity and/or size of the team. Additionally, if there is an outlier historical data point, such as an anomalously small Velocity value due to end-of-year holidays, remove it from the set of historical data to be simulated.

Monte Carlo simulation results are most useful in managing expectations with stakeholders. Product management typically wants to get as much output from an agile team as possible, while at times, losing sight of the distinction between outcomes over output. The output data grid from the tool provides very easily understandable Percent Chance and “Bet” values that should facilitate a conversation between the Scrum Master and Product Manager, or other stakeholder, who tends to plan based on the most optimistic projections of team output.