GuideUncertainty Quantification

Uncertainty Quantification: How to Put Honest Odds on a Development Plan

Uncertainty quantification replaces the single number in a development plan with a measured range, a probability of clearing the threshold that matters, and a ranked list of the unknowns driving your exposure.

November 19, 2026·72 min read·By the Incertive Team

Uncertainty quantification is the discipline of working out how wrong an estimate could be, and with what likelihood, for a quantity nobody can know exactly. Applied to a development plan - a product build, a capital programme, a software release, a market entry - it replaces the single number in the cell with something far more useful: a measured range, a probability of clearing the threshold that actually matters, and a ranked list of the unknowns driving your exposure. The plan stops being a prediction and becomes a bet whose odds you can read.

The gap between those two things is where most development plans are lost. A plan that says “eleven million and nine months” contains no information about how wrong it might be, in which direction, or which assumption would have to break for it to be wrong. A plan that says “a fifty percent chance of delivering inside nine months, an eighty percent chance inside eleven, and the integration effort is responsible for most of the spread” contains all three, and it does so without asking anyone to be a better forecaster. The difference is not optimism or pessimism. It is method.

This guide covers what uncertainty quantification means, the two fundamentally different kinds of uncertainty that live inside every plan, the evidence on what happens to plans that skip the exercise, the step-by-step workflow, the methods that do the quantifying, how to read percentiles and S-curves and confidence levels, why correlation between inputs is the mistake that makes every plan look safer than it is, how the practice changes through the development lifecycle, what to look for in software, the seven ways the exercise goes wrong, and how to make it stick in an organisation. The simulation engine underneath most of it is covered in depth in our complete guide to Monte Carlo simulation; this guide is about the surrounding discipline that makes the simulation worth running. Every figure cited below is linked to its source, and every number used to illustrate a hypothetical is labelled illustrative.

conceptdesignexecutionP90P50P10the plan numberOutcomeTime through the development plan
A plan states one number for every stage. The honest picture is a band that widens with the unknowns still outstanding, and the plan number usually sits near its optimistic edge (illustrative).

What uncertainty quantification actually means

Uncertainty quantification is the discipline of establishing, for a quantity you cannot know exactly, how wrong your estimate could be and with what likelihood. That is the whole of it. It does not claim to tell you what a programme will cost or when it will finish; it tells you which outcomes are consistent with what you currently know, and how much of the probability sits in each part of that range. For anyone who owns a development plan, the shift is smaller than the vocabulary suggests. You already know the schedule is not certain. Quantification simply insists that you say so in a form that can be argued with, budgeted against and checked afterwards, rather than in a footnote that reads “dates subject to change”.

It is also not a single technique, which is where teams usually get lost while hunting for the one tool that solves the problem. Quantification is a workflow with four steps, and each of them can be done well or badly. You identify what is uncertain. You characterise each uncertainty, giving it a plausible range and shape rather than one figure. You propagate those uncertainties through whatever model connects inputs to the outcome you care about, which for most plans means simulating the plan thousands of times instead of solving it once. Then you report the result in a form a decision maker can act on. Monte Carlo simulation is the standard engine for the third step, and our guide to Monte Carlo simulation for business covers that engine in depth; this post is about the discipline that surrounds it.

From a single number to a measured range

Compare two sentences about the same programme. The first says the build will cost 4.2 million and the team ships in nine months (illustrative). The second says there is roughly a 50 percent chance of shipping by nine months and an 80 percent chance by eleven, with cost following a similar shape (also illustrative). The first version is easier to put in a board pack and is almost always wrong, because a single number drawn from a wide range of possible outcomes is only ever right by coincidence. The second is harder to say out loud and considerably more useful, because it tells a reader which commitments are safe, which are aspirational, and roughly what it would take to make the aspirational ones safe.

Point estimates persist because organisations reward them. A figure with two decimal places reads as competence while a range reads as hedging, and the person offering the range is often the one sent away to come back with a number. The result is a plan whose precision is entirely manufactured: the 4.2 million was the midpoint of a disagreement, the nine months was the date the sponsor wanted to hear, and neither carries any information about how far either could move. We have written separately about the hidden costs of false precision, but the short version is that a single number conceals exactly the information a decision needs, which is how wrong the plan can be before the decision itself should change.

Public sector cost estimating has lived with this problem for decades, and its rulebook is explicit about the remedy. The US Government Accountability Office cost estimating guide treats risk and uncertainty analysis as a required step whose purpose is to attach a level of confidence to a point estimate rather than to discard the estimate. The guide notes that the analysis may use a three-point estimate methodology, a risk-driver approach, or both, and that a three-point method yields a contingency amount while giving no indication of which risks most affect programme cost. Its worked air-vehicle example makes the gap concrete: a point estimate of 2.47 billion dollars, a cost at the 50 percent confidence level of 2.61 billion, and 2.79 billion once the programme is budgeted to 80 percent confidence.

$320 million - separates a point estimate of $2.47 billion from the same air-vehicle programme budgeted to an 80 percent confidence level, in the worked example in the federal cost estimating guide.

Source: US Government Accountability Office

The measurement community states the underlying principle more bluntly than most planners would dare. NIST Technical Note 1297 holds that the value of a measured result is largely determined by the quality of the uncertainty statement that accompanies it: a figure with no credible account of its own error is close to useless to anyone who has to act on it. Translated into planning terms, the output of quantification is not a better single number but a described shape, which is why a probability distribution and S-curve carrying explicit P10, P50 and P80 points is the natural deliverable. A reader can then take a commitment off that curve at whatever confidence the situation deserves, instead of taking one number and privately discounting it by an amount nobody records.

Verification, validation and quantification are three different questions

Teams that start quantifying uncertainty quickly meet a question they did not expect: how do they know the model itself is any good? The 2012 National Research Council study on the reliability of complex models, requested by the Air Force Office of Scientific Research with Department of Energy support, separates three questions that are routinely confused with one another. Verification asks how well a computational model solves its underlying equations. Validation asks how well the model represents the real system it is meant to stand for. Uncertainty quantification asks how much the answer should be trusted, and it plays a role in both validation and prediction. The three are sequential in practice and independent in principle, which is why a model can be impeccably verified and still be completely invalid.

The translation into a development plan is direct and worth making explicitly. Verification is whether the arithmetic holds: whether the dependencies are wired the way the team believes, whether durations add correctly along the critical path, whether a cost line has been counted twice. That is the part a spreadsheet gets wrong silently, and the reason the limits of forecasting in Excel are worth understanding before anyone trusts an output. Validation is whether the structure of the plan resembles the way the work actually behaves, which is a judgement about the world rather than about the file. Quantification is the third question: given that the structure is roughly right and the arithmetic sound, how much confidence does the output deserve. Most planning arguments are really disputes about the second question conducted in the language of the first.

Mature engineering disciplines have formalised all of this, which should reassure anyone who suspects they are being asked to do something soft. ASME V&V 20-2009 is an ANSI-approved standard for verification and validation in computational fluid dynamics and heat transfer, and its stated objective is to quantify the degree of accuracy that can be inferred from comparing a solution with data at a specified validation point. The caveat attached to that objective is the instructive part: accuracy elsewhere in the domain remains a matter of engineering judgement. Validation under that standard is an assessment of model error, not a pass or fail test, and the comparison it licenses is local rather than universal.

That is a useful corrective for teams who expect a plan to come back stamped “valid”. No analysis certifies a plan. It bounds how wrong the plan might be, under stated assumptions, at the points where it has actually been checked. The same caution applies to where the inputs come from. NIST distinguishes Type A evaluation, derived by the statistical analysis of a series of observations, from Type B evaluation, derived by judgement from everything else available, including experience, specifications and comparable cases. The distinction matters in planning because almost every input in a development plan is Type B: there is no dataset describing how long this particular integration will take. That is not a disqualification, and uncertainty-first planning depends on it working, but it does mean the quality of the elicitation sets the ceiling on the quality of the answer.

What uncertainty quantification is not

A risk register is not uncertainty quantification, although the two are often filed under the same heading in the same governance pack. The register is genuinely useful: it forces a team to name what could go wrong, assign an owner to each item and revisit the list on a cadence, and plenty of failures trace back to a risk nobody ever wrote down. What a register cannot do is arithmetic. Forty entries scored high, medium and low tell you nothing about the probability of finishing by the committed date, because nothing in the document connects any entry to the outcome. Two registers can look identical while describing plans with completely different odds, and a register with everything marked medium is indistinguishable from one filled in at random.

A best, base and worst case model is a step closer and still not quantification. Three cases are a reasonable way to open a conversation, and the act of writing the worst case often surfaces an assumption nobody had stated aloud. The problem is that the three numbers are arbitrary points with no probabilities attached, so there is no way to say whether the base case is a coin flip or a long shot. Worse, the worst case is usually not the real worst case: it tends to be the base case with a little taken off, because a genuinely bad scenario is politically difficult to put on a slide. Structured three-point estimation at least attaches a shape to each input, which is how three numbers become a distribution rather than three opinions.

Sensitivity analysis on its own answers a different question from the one most people think they are asking. It tells you which inputs move the outcome most, which is exactly what you want when deciding where to spend management attention, and sensitivity analysis deserves a place in every serious plan review. But a ranking is not a probability. Knowing that supplier lead time is the largest driver of your launch date does not tell you how likely you are to launch on time, or whether the exposure amounts to two weeks or two quarters. The ranking and the probability answer each other: one says where to look, the other says how much trouble you are in. A tornado diagram with no distribution behind it is a list of suspects with no case attached.

A dashboard is not a forecast either. Business intelligence reporting is excellent at what it does, which is to describe what has already happened with accuracy and speed, and a team without it is blind to its own present. But a burn-down chart, a variance report and a cost-to-date figure are all statements about the past, and extending them forwards by eye is not quantification. The distinction matters because dashboards create a powerful feeling of control: the numbers are current, the charts are tidy and nothing on the screen is wrong. None of it answers the only question a sponsor really has, which is what the odds are that the remaining work lands inside the remaining budget and time.

Finally, a buffer is not an uncertainty statement. Contingency set by convention - ten percent on cost, two weeks for every quarter of schedule, a round number that survives because nobody has challenged it - may well turn out to be adequate, but it carries no evidence about the plan it is attached to and no information about the confidence it buys. The alternative is not a larger buffer; it is a buffer derived from the analysis, with the confidence level stated as a decision rather than inherited as a habit. Large public programmes work this way as a matter of policy, which is instructive precisely because they are not free to choose a comfortable number and move on.

70 percent - is the joint cost and schedule confidence level that NASA mission directorates plan and budget projects to at a life-cycle cost of $1 billion or more, with any lower figure justified in the decision memorandum.

Source: NASA NID 7120.122

The two kinds of uncertainty in a development plan

Not all uncertainty in a plan is the same kind of thing, and the single most useful sorting exercise a team can perform is to split its unknowns into two piles. The distinction is standard in reliability engineering, where Der Kiureghian and Ditlevsen set out the usual reference treatment in Structural Safety in 2009. Their point is narrower and far more practical than the Latin suggests: the division turns on whether the modeller expects to be able to reduce the uncertainty. Aleatory uncertainty is the part that will still be there after any amount of study. Epistemic uncertainty is the part that can be cut by collecting more data or refining the model. The two coexist in every real plan and should be kept distinct, because they call for different kinds of money.

Aleatory uncertainty: the variation you cannot remove

Aleatory uncertainty is the irreducible scatter in how the work actually happens. A crew that averages eleven units a day does not deliver eleven units every day; it delivers nine, then fourteen, then eleven, and the variation is a property of the process rather than a gap in your knowledge of it. The same applies to weather on a site, to which specific customers in a qualified pipeline happen to convert this month, to component yield on a line running comfortably inside specification, and to how many support tickets arrive in a given week. You can measure these patterns, and with enough history you can measure them well. What you cannot do is make any of them collapse to a point.

This is the uncertainty that most reliably surprises teams, precisely because each instance of it is small. Nobody plans a programme around one bad day. The difficulty is that a long chain of modest, independent variations does not average out neatly once the tasks are linked: a dependent task cannot start early merely because its predecessor finished late, so delay propagates while early finishes are quietly absorbed. That asymmetry is why plans built from average durations finish late far more often than they finish early, and why the honest way to represent throughput is as a range with a shape, which is what probabilistic forecasting delivers and what a deterministic bar chart cannot.

Because it cannot be removed, aleatory uncertainty has to be absorbed, and the instruments for absorbing it are physical and financial rather than analytical: schedule float, cost contingency, inventory, standby capacity, a second qualified supplier, an operator who can be pulled across from another line. Each of those costs money and buys protection. Commissioning another study buys nothing, because there is nothing further to learn. Quantification is how the amount gets sized rather than argued: simulating the plan across thousands of runs shows how much buffer is required to reach a stated confidence, which converts a dispute about whether the contingency looks generous into a statement about what level of protection a given sum actually purchases. Monte Carlo simulation exists largely to answer that one question well.

Epistemic uncertainty: the ignorance you can buy down

Epistemic uncertainty is ignorance, and ignorance has a price list. Whether the integration between two systems will work at all, how a regulator will rule on a filing, what a supplier will actually charge once the specification is final, what is genuinely under the site, whether the market wants the thing you are building: none of these is a random process that will scatter politely around a mean. Each has a single true answer that you do not currently know. That is why the ranges attached to them are often enormous compared with the tidy variation of a production line, and why they dominate the outcome of early-stage plans while barely appearing anywhere in the schedule arithmetic.

The defining feature of this second pile is that study genuinely shrinks it. A technical spike resolves whether the integration works. A pilot in one region tells you something real about conversion. A letter of intent converts a guess about demand into a commitment you can point at. A soil survey replaces a range with a measurement. A firm quotation closes a cost question that could otherwise swing by a third. None of these is free and none is instantaneous, which is the whole reason the distinction is worth drawing: when an uncertainty is reducible, you can weigh the cost of reducing it against the cost of being wrong, and that comparison is a decision a sponsor can actually take.

It is also where the tail lives. The projects that end careers rarely fail because throughput varied; they fail because something assumed to be true was not, and the assumption was never tested because it never appeared as a number anywhere in the plan. Flyvbjerg and Budzier found exactly this pattern across roughly 1,500 IT projects: most overran moderately, while a minority went so far wrong that the distribution of outcomes looks nothing like a bell curve. Naming those assumptions as uncertainties, with ranges wide enough to be honest, is the only way they enter the arithmetic at all, which is why automatic uncertainty identification from a plain-language description of a plan matters more than it sounds.

One in six - of roughly 1,500 IT projects studied was a black swan, with a cost overrun of about 200 percent and a schedule overrun of about 70 percent.

Source: Harvard Business Review

Why the distinction changes what you do next

The practical payoff is a rule simple enough to apply in a meeting. Aleatory uncertainty is managed with buffer, float, inventory and capacity, because no amount of study removes it. Epistemic uncertainty is managed with information: a prototype, a pilot, a letter of intent, a survey, a technical spike, because study genuinely shrinks it. Get the two the wrong way round and both kinds of money are wasted. A plan that pads where it should investigate spends real cash carrying a risk it could have retired for a fraction of the amount, and carries it anyway. A plan that investigates where it should pad produces an elegant analysis of variation it was never going to remove, and still misses its dates.

Sorting a plan’s unknowns into these two buckets is often the single most useful hour a team spends, and it needs no software to begin. Take the list of things that could move the outcome and ask one question of each: if we spent a month and a reasonable sum finding out, would the range get narrower? If the answer is yes, the item belongs in the information pile and should be given an owner, a cost and a date. If the answer is no, it belongs in the absorption pile and should be given a buffer sized by analysis rather than by habit. The exercise also tends to shorten the list, because several entries on a typical register turn out to be the same underlying unknown wearing different labels.

The split also tells you which uncertainties are worth paying to reduce before a gate and which simply have to be carried into it. An uncertainty that is reducible, material to the decision and resolvable inside the available time is a candidate for an investigation funded now. One that is reducible but cannot be resolved before the gate has to be carried explicitly, with the decision framed around what happens either way rather than around a hope that it resolves kindly. And some base rates are simply the environment you operate in, and will not narrow for anybody, however diligent the team.

About 20% - of new business establishments fail within their first year, and about half are gone by year five, a base rate that no amount of internal planning rigour narrows.

Source: US Bureau of Labor Statistics

Where quantification reaches its limit

Honesty about the method requires saying where it stops. Some uncertainties resist quantification entirely. A genuinely novel technology has no reference class and no frequency data behind it, so any distribution placed on its performance is invention dressed as analysis. A competitor’s unannounced strategy is not a random variable; it is a decision being taken by people with a plan of their own. A change of government, a new regulatory regime, a war, a pandemic: none of these is the tail of a distribution anybody actually modelled, and treating them as though they were is how false precision gets smuggled back in at the very end of an otherwise careful process.

The right response is to say so rather than to invent a number. Quantifying what can be quantified while naming what cannot is more rigorous than quantifying everything, not less, because it tells a reader precisely which parts of the answer rest on arithmetic and which rest on judgement. A short, explicit list of unmodelled assumptions attached to a probability of success is worth considerably more than a tidier figure with the same assumptions buried inside it. The alternative, a model that silently absorbs the unquantifiable into a plausible-looking range, manufactures a confidence that nothing in the analysis has earned, and it is the version that tends to survive contact with a board.

That leads to the constraint governing all of this: a model’s output is only ever as good as the judgement in its inputs. Simulation creates no information. It propagates what you give it, faithfully, including your optimism, and a confident curve built on three badly elicited ranges is a well presented guess. The defences are unglamorous: structured three-point estimation rather than a single figure, ranges wide enough to be slightly embarrassing, inputs attributed to whoever supplied them, assumptions written where a reviewer can disagree with them, and a method that states its own limitations. Incertive publishes its methodology including those limitations, for the straightforward reason that a reader who cannot see where a model stops has no way to judge what it says.

Aleatoryirreducible variationrepeated outcomesCarry it: float, buffer,capacity, inventoryEpistemicreducible ignoranceno datapilot runcontractedBuy it down: prototype,pilot, survey, quotation
Sorting a plan’s unknowns into variation you cannot remove and ignorance you can pay to reduce is what tells you where to add buffer and where to buy information instead (illustrative).

What happens to development plans that skip uncertainty quantification

The case for quantifying uncertainty in a development plan does not rest on statistical taste, and it does not have to be argued from first principles. It can be argued from the record. Large projects have been studied for decades, across industries, countries and funding models, by people with no particular stake in the answer, and the studies keep returning the same verdict: plans built as single numbers are beaten by reality in one direction, repeatedly, by margins that would be unacceptable anywhere else in a business. What follows is that record, and then the uncomfortable inference it supports about where the error actually comes from.

The overrun record is remarkably consistent

Start with the broadest sweep. McKinsey’s review of major capital projects, covering more than 500 of them, puts the average cost overrun at roughly 79 percent. That figure deserves a moment of attention, because the instinct on first reading is to assume it describes a tail of disasters. It does not. It is the average, which means that for every project delivered close to plan there is another that roughly doubled, and that the central tendency of the whole population sits most of the way to twice the approved budget. No organisation approves a plan believing that is the likely outcome, and yet that is the outcome the population produces.

79% - average cost overrun across a review of more than 500 major capital projects.

Source: McKinsey & Company

Narrow the lens to construction and the picture sharpens rather than softens. McKinsey’s work on the sector’s digital future records that large construction projects typically run up to 80 percent over budget and take about 20 percent longer than scheduled, a combination that quietly compounds: a schedule slip carries its own financing, supervision and escalation costs, so time and money are not two independent failures but one failure counted twice. The KPMG Global Construction Survey found that only 31 percent of respondents’ projects came within 10 percent of budget, which is the same result stated from the other direction. Roughly two thirds of projects missed a tolerance band most boards would consider generous.

31% - of surveyed owners’ projects came within 10 percent of their budget.

Source: KPMG Global Construction Survey

The academic record agrees. Bent Flyvbjerg’s Iron Law of Megaproject Management states the finding with deliberate bluntness: megaprojects come in over budget, over time, over and over again, and nine out of ten of them run over budget. The measurement work behind it, by Flyvbjerg, Holm and Buhl, examined 258 transport infrastructure projects and found an average cost overrun of about 28 percent, which on its own looks almost mild beside the construction figures. The interesting part is the breakdown by mode: roads averaged about 20 percent, bridges and tunnels about 34 percent, and rail about 45 percent. The same estimating culture, applied to different physical problems, produced systematically different degrees of error.

258 - transport projects studied, with cost overruns averaging about 28 percent overall and about 45 percent for rail.

Source: Flyvbjerg, Holm and Buhl

What should unsettle a reader is not the size of any single number but the consistency of their sign. If estimating were merely difficult, the errors would scatter: some projects would land well under budget, and the averages would wash towards zero. They do not. Across roads, rail, tunnels, buildings and programme types, across public and private sponsors, and across study periods separated by decades, the error points the same way. One-directional error at that scale is the signature of a method problem rather than a skill problem, and we have traced the pattern at length in why projects fail and how to beat the odds and in our treatment of capital project risk.

Large technology programmes fail in the same shape

Software and technology programmes ought to break the pattern. There is no ground condition to discover, no land to acquire, no concrete to cure and no weather. Whatever drives the construction record should be absent, or at least muted, when the deliverable is code. The largest study of the question says otherwise. McKinsey, working with the University of Oxford, examined 5,400 large IT projects and found that 45 percent ran over budget, 7 percent ran over time, and the portfolio delivered 56 percent less value than had been predicted. The shape of the failure is the same as in construction. Only the proportions move.

45% - of 5,400 large IT projects ran over budget, and the portfolio delivered 56 percent less value than predicted.

Source: McKinsey & University of Oxford

The value figure is the one most often skipped, and it is the most consequential of the three. A programme can be delivered near budget and close to schedule and still be a failure, because the benefit case that justified it never arrives. Measuring only cost and time therefore flatters the record: it scores the inputs and ignores the product. When the same study reports that 17 percent of large IT projects go so badly that they threaten the existence of the company, it is describing something no cost variance column can capture. Those are not projects that overspent. They are projects that consumed the organisation which commissioned them.

The Harvard Business Review analysis by Flyvbjerg and Budzier, drawing on roughly 1,500 IT projects, gives that distribution its shape. The average cost overrun was 27 percent, a figure comfortable enough to be absorbed into most contingency lines without much argument. But one project in six was what the authors call a black swan: a cost overrun of around 200 percent and a schedule overrun of around 70 percent. Those few cases carry most of the aggregate damage, and they are entirely invisible in the average. An organisation that plans against the 27 percent is well prepared for the ordinary case and wholly unprepared for the case that actually threatens it.

1 in 6 - IT projects studied was a black swan, with around 200 percent cost overrun and around 70 percent schedule overrun.

Source: Harvard Business Review

This is where the two literatures converge on a single statement about plans. The distribution of project outcomes is not symmetric. There are a small number of ways for a development plan to finish early and under budget - a dependency clears faster than expected, a supplier beats its quote, a scope item turns out to be unnecessary - and a very large number of ways for it to finish late and over. The ways of going wrong also compound, because delay in one place consumes float everywhere else. The resulting distribution has a short left side and a long right tail, and a point estimate sits near its optimistic end, which is the mechanism behind what we have called the hidden costs of bad business decisions.

Why planning to the average overrun does not work

A reader who accepts all of this usually arrives at the same practical conclusion first: if projects of this kind overrun by some characteristic amount, then add that amount. Take the estimate, apply an uplift, approve the uplifted figure, and the historical error is neutralised. That is a genuine improvement on doing nothing, and it is the honest instinct of anyone who has read the evidence above. It is also, in four distinct ways, the wrong correction, and the ways it fails are worth setting out carefully, because each of them points towards what the right correction has to look like.

The first problem is that an average conceals the distribution it came from. Adding a mean overrun to a point estimate produces another point estimate, now positioned somewhere near the middle of the range of plausible outcomes rather than near its optimistic edge. That is better, but a figure near the middle of a range is, by construction, exceeded in something close to half of cases. Funding a plan at that level is a decision to be short of money roughly as often as not, which is rarely what anyone intends when they approve a budget and never what they tell the board they are doing. The uplift has changed the number without changing its nature: it is still one value standing in for a range, and it still carries no statement of confidence.

The second problem is that the right uplift is not one number. The transport figures make this unavoidable: roads at about 20 percent, bridges and tunnels at about 34 percent and rail at about 45 percent are not variations around a common value, they are different populations. A blended figure applied to all three is too generous for roads, roughly right for nothing, and far too thin for rail. McKinsey’s separate work on megaprojects reports rail cost overruns averaging about 44.7 percent alongside demand that was overestimated by 51.4 percent, which carries the same lesson over to the benefit side of the case: the uplift a plan needs depends on what kind of plan it is, and on which quantity is being uplifted.

The third problem matters most to anyone who has to manage the plan rather than merely fund it. A flat uplift is pure contingency: it says the plan may cost more without saying which parts of it are responsible. It therefore buys no management information whatsoever. It does not tell you whether the exposure sits in a permitting dependency, a single long-lead supplier, a hiring ramp or an adoption assumption, so it cannot be used to decide what to investigate, what to renegotiate or what to de-risk before committing. A ranked view of which variables actually move the outcome is the output that answers those questions, which is why we treat it as a primary result rather than an appendix; for readers coming to this cold, project risk management for non-technical leaders is the gentler entry point.

The fourth problem is the black swan finding itself. If one project in six overruns by around 200 percent, and if 17 percent of large technology programmes can threaten the company that commissioned them, then the quantity an organisation most needs to understand is not the central tendency of its exposure but the behaviour of the tail. An uplift based on the mean is, by definition, silent about the tail. What is needed is therefore not a bigger number but a different kind of answer: a distribution, from which any confidence level can be read off, and against which the question “how much would it take to be comfortable” has a real answer rather than a negotiated one. That is the subject of our guide to Monte Carlo simulation for business, and it is where the rest of this piece leads.

Why single-number plans survive anyway

If the evidence is this consistent and this public, the obvious question is why the single-number plan survives at all. The answer is not that practitioners are unaware of it. Most people who build plans for a living can recite the overrun statistics, and will tell you privately that their own headline figure is softer than it looks. The practice persists because three separate forces hold it in place: a cognitive one, a social one and a technical one. They reinforce each other, which is why addressing any one of them on its own tends not to work for long.

Optimism bias and the inside view

The cognitive force is optimism bias, and the particular form it takes in planning is established enough to have its own name: the planning fallacy. Teams estimate from the inside. They take the plan they intend to execute, decompose it into phases and work packages, estimate each piece, and sum the pieces. The procedure feels rigorous because it is detailed, and detail is easily mistaken for accuracy. But what it produces is an account of how the work will go if it goes as described, and it contains no mechanism at all for representing the possibility that the description itself is wrong.

Summing the parts also quietly removes the interactions that cause most real overruns. Each component is estimated on the assumption that it starts when it is supposed to and receives the attention it needs; the plan as a whole then assumes that all of those conditions hold simultaneously. Dependencies, shared scarce specialists, contention for a single test environment, the second-order effect of one late input on four downstream tasks: none of these live inside any individual estimate, so none of them appear in the total. The result is a number that is internally consistent and externally optimistic, which is the hardest kind of wrong number to challenge, because every line of it can be defended.

The correction is the outside view, usually implemented as reference-class forecasting: instead of asking what this plan should cost, ask what comparable plans actually cost. It is an unglamorous method and it is reliably effective, because it replaces a story about the future with a distribution drawn from the past. Flyvbjerg’s Iron Law exists precisely because the inside view is the default nearly everywhere, and the Harvard Business Review analysis of IT programmes reaches the same conclusion from the technology side. We have written up the mechanics in reference-class forecasting and the psychology behind it in optimism bias in business.

False precision as a social contract

The second force is social, and in most organisations it is stronger than the first. In a boardroom a range reads as weakness. A sponsor who answers “between 14 and 23 months, most likely around 17” (illustrative) sounds less in command of the facts than one who answers “18 months”, even though the first answer contains strictly more information and the second is almost certainly a selection from inside it. Organisations reward the estimate that is most confidently stated rather than the one most likely to be right, and they manage to do so without anyone ever deciding to.

The structure of the process makes that preference concrete. A budget line has one cell. An approval template has one figure for cost and one date for delivery. A board pack has a summary slide with a number on it, and a stage-gate paper carries a recommendation that has to be either yes or no. At every point where a plan is written down for other people, the format admits exactly one value per quantity. A team that has done careful probabilistic work still has to choose something to type into the box, and the moment they do, the analysis behind it stops travelling with it.

What emerges is a quiet bargain that almost everybody recognises. Everyone in the room knows the number is uncertain. Nobody is permitted to say so, because saying so sounds like hedging, and hedging sounds like a lack of conviction. So the estimator protects themselves with private padding that is never declared, the approver assumes some padding exists but cannot tell how much, and the organisation ends up holding a figure whose true confidence level is unknown to every single person who touched it. The padding is simultaneously real and undocumented, which is the worst of both available arrangements.

That bargain is expensive in a specific way: it removes exactly the information a decision needs. Without a distribution you cannot size contingency except by convention, you cannot compare two options on anything beyond their point estimates, and you cannot judge which assumption is worth spending a month investigating before you commit. The remedy is not to present a vague range in place of a number, it is to present a number with a stated confidence attached, which is both decisive and honest. That distinction is the subject of the hidden costs of false precision, and making it normal is a cultural task as much as a technical one, which is why we treat it under building a risk-aware culture.

The spreadsheet cannot hold the answer

The third force is technical, and it is the least discussed of the three. A deterministic model cannot represent a range, because a cell holds one value. This is not a shortcoming of any particular tool; it is a property of the representation. The moment a plan is expressed as a grid of single values multiplied and summed, the output is a single value, and whatever uncertainty existed in the analyst’s head at the moment of entry has been discarded before any arithmetic takes place. Everything downstream inherits that loss, and no amount of care further down can restore it.

What practitioners do instead is swap assumptions one at a time, or build a best case, a base case and a worst case as three parallel columns. Both moves are reasonable and both are far weaker than they appear. Varying one input while holding the rest fixed explores a vanishingly small share of the combinations that actually occur, because real outcomes come from several inputs moving together. And a three-column layout produces three stories rather than a likelihood: nothing in it indicates whether the worst case has a one-in-three chance or a one-in-fifty chance, which is the only thing a decision maker needs in order to act on it.

  • A best case and a worst case built by pushing every input to its own extreme describe outcomes so improbable that neither is a usable planning target.
  • Changing one assumption at a time cannot reveal the combinations that matter, since the damaging cases usually involve two or three things going wrong together.
  • A scenario column carries no probability, so it cannot be weighted, compared against a rival option, or used to justify a particular contingency figure.
  • Linked assumptions are typically entered independently, so the model can quietly combine a pessimistic schedule with a cost that could never occur alongside it.

None of this is an argument against spreadsheets, which remain the right instrument for a great many things. It is an argument that the question of how likely a plan is to succeed is not one of them, and that the workarounds people adopt are symptoms of putting a probabilistic question to a deterministic model. We have catalogued the specific ways that breaks down in the limitations of forecasting in Excel. The alternative is to describe the plan and its uncertainties once, in plain language, and then let simulation do the combinatorial work that a one-cell-per-assumption model structurally cannot.

The cost of being confidently wrong

It is worth remembering how routinely plans fail to survive contact with the world, including at the smallest scale. United States Bureau of Labor Statistics data on new business establishments shows that about 20 percent do not last their first year, and about half are gone by year five. Every one of those ventures began with a plan, and almost all of those plans were expressed as single numbers: a revenue figure, a break-even month, a hiring schedule. The base rate of plans not working out is simply high, and no quantity of internal conviction at the moment of approval moves it.

About 50% - of new business establishments in the United States are no longer operating by their fifth year.

Source: US Bureau of Labor Statistics

At the other end of the scale the stakes change in character rather than in degree. The McKinsey and Oxford study of large IT projects found that 17 percent of them went so badly that they threatened the existence of the company that had undertaken them. That is not a budget variance; it is an existential event arriving through a programme approved on the strength of a plan nobody had stress-tested probabilistically. The organisations involved were not reckless. They approved a number that looked firm, and the firmness turned out to be an artefact of the reporting format rather than a property of the world.

17% - of large IT projects go so badly that they threaten the existence of the company.

Source: McKinsey & University of Oxford

The damage from a confidently wrong plan is also larger than the overrun it produces, because the overrun is merely the part that gets measured. The more expensive loss is the set of decisions that were never revisited. A plan that looks certain does not invite a second option, so the phased alternative never gets costed. It does not invite an investigation, so the assumption that later proves fatal is never tested while testing is still cheap. And it does not invite a stopping rule, so the programme clears each gate on momentum rather than on evidence, and by the time the trouble is undeniable the sunk cost is doing the arguing.

Quantifying uncertainty does not make any of those outcomes impossible. It makes them visible early enough to act on, which is a narrower and far more achievable claim. A plan that arrives with a probability of success, a ranked list of the variables driving that probability, and a comparison against an alternative such as a phased rollout gives a board something to decide with, rather than a single figure to either wave through or send back. That is the whole of the shift being argued for here: the same plan, carrying its own uncertainty, stated as a confidence rather than asserted as a fact.

-10%plan+5%+10%+15%+20%+25%+30%+40%+50%Approved budgetWithin budgetOver budgetSimulated total cost, relative to the plan value
Propagating the input ranges produces a distribution rather than a number. The share of the mass to the right of the budget line is the probability of a breach (illustrative).

How to quantify uncertainty in a development plan, step by step

Uncertainty quantification sounds like a modelling project, which is why so many plan owners outsource it, postpone it, or quietly decide it does not apply to them. In practice it is a workflow: seven steps, run in the same order every time, that turn a plan expressed in single numbers into a plan expressed in ranges and probabilities. None of the steps requires a statistician. Each one is a judgement about the business: what the decision actually turns on, which assumptions are shaky, how wrong they could plausibly be, and what you would do differently if the odds came back worse than you hoped. The arithmetic in the middle is mechanical and can be handed to software. The judgement at either end cannot, and it is the part a plan owner is already qualified to supply.

The sequence matters more than the sophistication. A crude range on the three inputs that actually drive the outcome will beat an elaborate model built on inputs nobody questioned, because the error that sinks a development plan is almost never a rounding error: it is a structural assumption that was never examined. Teams who work through the steps below usually find the first pass takes an afternoon and the second takes an hour, and that most of the value arrives before the output does, in the arguments the exercise forces into the open. Someone has to say out loud how late the permit could be, or how thin the pipeline really is, and the act of writing a range beside that assumption is often the moment the plan changes.

Step 1: define the output the decision actually turns on

Most quantified plans measure the wrong thing. A board pack full of distributions on headcount, unit cost and conversion rate can still fail to answer the only question in the room, because nobody named the quantity the decision turns on. So force that nomination first. Write down one number: total cost to complete against the approved budget, the delivery date against the committed launch, net present value against the hurdle rate, payback period against the window the business can tolerate, or the peak cash requirement against the facility available. If the plan owner cannot state that quantity in a sentence, the analysis has no target, and it will drift towards whatever is easiest to model rather than whatever is at stake.

The threshold is not decoration; it is the thing that makes the output actionable. A distribution on its own is a shape, and it becomes a decision input only once a line is drawn across it, because the useful answer is never the shape of the curve but the probability of clearing the line. “Our cost estimate has a standard deviation of eleven percent” changes nothing. “There is a 38 percent chance of delivering inside the approved budget” (both figures illustrative) changes the conversation immediately, and it changes it in a way everyone present can argue with. Name the line before you run anything, and name it in the units the commitment was made in rather than the units that happen to suit the model.

Real plans usually carry two or three of these quantities rather than one, and they interact. A development programme is typically committed on cost and on date, with a commercial return hanging off both. Those outputs are not independent: the cheapest way to hold a date is usually to spend more, and the cheapest way to hold a budget is usually to let the date slip, so a plan that looks comfortable on each output read separately can be distinctly uncomfortable on the pair. Public practice recognises this explicitly. NASA policy for its largest projects is built around a joint confidence level, a single probability covering cost and schedule together, precisely because reporting the two in isolation flatters the plan.

Expect this step to take minutes and to be contested anyway. Disagreement here is informative: if the finance lead names peak cash while the programme director names the launch date, the organisation has just discovered it has been running two different plans. Settle the primary output, note the secondary ones, and record the thresholds in writing. Everything downstream, including which inputs are worth ranging at all, depends on this choice, which is why it is the one step you should never let the software make for you.

Step 2: find the uncertain inputs

A development plan of any size contains hundreds of assumptions and perhaps six that matter. The purpose of this step is to separate them, and the fastest way is to walk the plan’s own logic and ask of every line a single question: if this were twice as bad, would the answer change? Most assumptions fail that test instantly. Office costs could double without moving a programme verdict; the integration effort doubling could sink it. The question is deliberately crude, because crudeness is what makes it fast, and speed is what lets you apply it across the whole plan rather than only to the part someone already suspected.

Resist the instinct to be exhaustive. An over-specified model is harder to trust, harder to review and no more accurate, because the spread of an output is governed by the inputs that dominate it, not by the number of inputs included. Thirty ranged variables, most of them inert, obscure the four that drive the result and invite reviewers to argue about the wrong ones. They also cost something real: every range you add is a judgement somebody has to defend, and judgement is the scarce resource in this exercise. Two genuinely uncertain, genuinely influential drivers, characterised with care, are worth more than twenty specified casually, and the ranking step later will tell you whether you missed one.

The inputs people forget are almost always the ones outside their own function. Engineering teams range the build and treat the approval as a date; commercial teams range the pipeline and treat hiring as instantaneous. Yet the dependency that most often decides a development plan is not the work itself but the thing that has to happen before the work can start: the permit or regulatory clearance that has to be scheduled rather than decided, the supplier that has to be contracted on terms nobody has tested, the two senior hires the plan assumes are in post in month one. Those items have long tails and no internal owner, which is exactly the combination that produces surprises. Walk the plan with someone from a different function and the list grows in the right direction.

The practical obstacle at this step is the blank page rather than the thinking. Staring at a plan and asking what could go wrong is a task people perform badly under time pressure and far better from a first draft, which is why Incertive starts from a plain-language description of the plan and automatically identifies the uncertainties and risk factors inside it, producing a first list to argue with instead of an empty column. Treat that list as a starting point and not a verdict: add what no tool could see, strike what does not matter, and keep the version you are willing to defend in front of the people who will be held to it.

Step 3: characterise each input as a range

This is where most of the quality lives, and where most teams either do the work or quietly skip it. The workhorse technique is three-point estimation: for each driver, a minimum, a most likely and a maximum. It dominates not because it is elegant but because it is answerable. Ask an experienced engineer for the standard deviation of an integration effort and you will get a shrug; ask for the fastest it could credibly go, the figure they would plan on, and the point at which they would start apologising, and you will have three usable numbers inside a minute. Our guide to three-point estimation works through the mechanics, and the PERT calculator turns one set of three figures into an expected value and a spread with no setup at all.

Federal cost estimating practice treats this as settled method rather than a shortcut. The guidance describes a risk and uncertainty analysis as something that can be conducted with a three-point estimate methodology, with a risk-driver approach that models discrete risks and their effects, or with a combination of both, and it names the distributions cost estimators reach for most often. Knowing that the menu is short is itself useful: the choice of distributional family is a second-order question next to whether the range is honest, and a team agonising over beta against lognormal before it has pressure-tested its maximum is optimising the wrong thing.

Five - distributions - triangular, lognormal, beta, uniform and normal - are the ones cost estimators most commonly use when characterising an uncertain input.

Source: GAO-20-195G

The commonest objection at this step is that there is no data. Metrology answered that objection decades ago and gave it a vocabulary. The standard reference separates Type A evaluation, where an uncertainty is derived statistically from a series of observations, from Type B evaluation, where it is derived from everything else: earlier measurements, experience, specifications, published information and informed judgement. Both are legitimate, both end up expressed in the same units, and both combine into a single statement of uncertainty. A development plan is overwhelmingly Type B territory, since almost nothing in it has been done before under the same conditions. That is a reason to be careful about how judgement is elicited, not a reason to decline to quantify, and a discipline that reports uncertainties on physical constants manages it without embarrassment.

95% - is approximately the level of confidence carried by an interval formed with a coverage factor of k = 2, in the formal vocabulary used to report an uncertainty alongside a measured result.

Source: NIST Technical Note 1297

Elicitation technique matters more than people expect, because the ranges you collect are only as good as the order in which you ask for them. Ask for the extremes first and the most likely last: if the plan number is on the table at the start, every other figure is dragged towards it, and you will end up with a range that is decoration on the original estimate. Define what the endpoints mean, out loud, before anybody answers. The minimum is not the best imaginable outcome and the maximum is not the worst nightmare; they are values you would be genuinely surprised to see beaten, which in practice means something like a one-in-ten or one-in-twenty event rather than a physical limit. Without that definition, two experts hand you numbers drawn from two different mental scales and the model quietly averages them.

Ask different people separately and reconcile afterwards. A range agreed in a meeting is usually the range held by the most senior person in the meeting, and the spread between independent answers is itself a finding: when two people who both know the work give maxima that differ by a factor of three, the disagreement matters more than either number. Finally, apply one blunt sanity check to every range before accepting it. Is the plan value sitting at the optimistic edge of it? If the plan says twelve weeks and the range runs from eleven to fourteen (illustrative), nothing has been stress-tested, because the asymmetry in real development work runs the other way: tasks are bounded below by the work itself and effectively unbounded above by everything that can interrupt it.

Step 4: propagate the uncertainty through the plan

Propagation is the step people imagine is the hard one, and it is the only step that is genuinely mechanical. With every driver described as a range rather than a point, the plan is no longer a single calculation; it is a calculation that can be performed many times over. Each run draws one value from each input range, pushes those values through the plan’s own arithmetic, and records the resulting cost, date or return. Do that a few thousand times and the recorded outputs accumulate into a distribution: a picture of the plan’s possible results together with how often each one occurred. Nothing more conceptual than that is taking place, and no part of it is beyond a spreadsheet in principle, though the bookkeeping becomes tedious quickly.

This is why Monte Carlo simulation became the standard engine rather than one option among several. Analytical methods for combining uncertainties require the model to behave itself: additive structure, well-mannered distributions, no awkward logic. Development plans do not behave. They contain thresholds, conditionals, staged gates, capacity limits and knock-on effects where a slip in one workstream idles another, and a simulation imposes no restriction on any of it, because it never has to understand the model, only to run it. Our guide to Monte Carlo simulation for business covers the method end to end, and the Monte Carlo calculator is enough to watch the mechanics work on a small plan before committing to anything larger.

Be clear about what comes out, because this is where expectations usually go astray. The output of propagation is not a better single number. It is the full set of plausible outcomes with their frequencies, and its value is that any probability question can be read off it afterwards without running anything again. What is the chance of finishing inside budget? What figure covers us eight times in ten? How much worse is the ninetieth percentile than the plan? Those are three questions about one object, and expressed as a cumulative curve that object answers all three by inspection, which is why the S-curve rather than any single percentile is the artefact worth putting in front of a decision-maker.

Step 5: rank the drivers

A distribution tells you how exposed you are. It does not tell you why, and on its own it offers nothing to act on: a plan with a 40 percent chance of clearing its budget and a plan with a 90 percent chance (both illustrative) leave the obvious question equally unanswered, which is where the exposure comes from. Sensitivity analysis answers it by asking, systematically, how far the output moves when each input travels across its own range. The result is a ranking rather than a number, and the ranking is what converts an analysis into a management agenda, because a ranked list has owners and next actions attached to it in a way that a percentage never does.

The conventional presentation is the tornado diagram: one horizontal bar per input, showing how far the output travels as that input sweeps its range, sorted with the longest bar at the top so the shape narrows downwards. It reads in seconds and it is almost impossible to misinterpret, which is rare in this field. Two properties push an input towards the top, and both are necessary. The first is how far it can swing, which is the range you set in step three. The second is how strongly the output responds, which is a property of the plan’s structure. An input with a huge range that the plan barely depends on is harmless; an input with a narrow range sitting on the critical path can dominate everything. We walk through reading these charts in sensitivity analysis explained, and Incertive ranks the variables that most affect the outcome and presents them as a tornado diagram.

Ranking is a separate and necessary step, not a refinement of the previous one. Federal cost estimating guidance makes the point sharply against the very method it recommends: a three-point approach yields a contingency amount, but that figure on its own gives no indication of which risks are most likely to affect program cost. A contingency answers how much; it is silent on what, and a plan owner handed a number with no attribution can do nothing with it except hold it or spend it. The ranking restores the attribution, and that is the difference between knowing a plan is risky and knowing what to do about it on Monday morning.

Read the ranking as a shopping list for where to spend money reducing uncertainty, because reducing uncertainty costs money and the budget for it is finite. The top two or three bars are the only places where a prototype, a pilot, a survey, a fixed-price contract or a second quote will materially change the plan’s odds; spending anywhere further down the list buys precision nobody needed. That is also the honest test of whether the analysis earned its keep. If the tornado told you only what you already knew, you have lost an afternoon. If it put something at the top that nobody had been actively managing, it has already paid for itself several times over.

Step 6: turn the distribution into a decision

Here is the step most analyses skip, and skipping it is why so many quantified plans end their lives as an appendix. The conversion is simple and it has to be done deliberately: read the probability of clearing the threshold you named in step one, decide what confidence level you are willing to fund or commit to, and compare the two. The second half of that is a policy choice rather than a modelling output, and it belongs to whoever carries the consequence. Large public programmes make the choice explicitly and write it down, and that is the practice worth borrowing even at a tiny fraction of the scale.

$320 million - separated the point estimate from the 80 percent confidence figure in the air vehicle example in federal cost estimating guidance, where a $2.47 billion point estimate required $2.79 billion to budget at 80 percent confidence.

Source: GAO-20-195G

The level itself is a judgement about appetite, not a universal constant. NASA policy for projects and single-project programmes with a life-cycle cost of a billion dollars or more directs mission directorates to plan and budget based on a 70 percent joint confidence level or as approved by the decision authority, and requires that any justification for budgeting below that level be recorded in the decision memorandum. A product team needs neither a seventy percent threshold nor a decision memorandum, but it does need the two habits underneath them: a stated level, chosen before the number arrives, and a written reason on the rare occasions a plan is funded below it.

There are three real answers at this point, not two. Proceed, because the probability clears the level you set. Decline, because it does not and nothing available moves it. Or proceed conditionally, once named actions have shifted the odds: the supplier contract signed before the long-lead order, the two hires closed before the build phase opens, the pilot run before the full launch is funded. The third answer is the most common and by some distance the most valuable, because it is the only one that uses the analysis to change the plan rather than merely to grade it. Incertive returns a go, no-go or conditional recommendation alongside the probability of success, and the conditional verdict is the one worth reading closely, because it names what has to become true.

The same machinery lets you price a mitigation before buying it, which is the most underused capability in the whole workflow. A mitigation is, in model terms, a change to a range: a fixed-price contract truncates the upper tail of a cost input, a pilot narrows a demand range, an early hire removes a start-date dependency. Make the change, run it again, and look at what happens to the probability of clearing the threshold. Sometimes a measure everybody assumed was essential moves the number by two points while costing a quarter of the contingency, and sometimes an unglamorous change to sequencing moves it by fifteen (both illustrative). Neither fact is visible without the model, and both are ordinary management decisions once it exists.

Step 7: record the forecast so it can be scored later

The last step is the one organisations abandon most readily, because its benefit arrives long after the decision does. A forecast that is never compared with the outcome cannot teach anybody anything. Worse, it leaves a team free to remember its estimates as having been roughly right, which is the mechanism by which the same error survives for a decade: the plan slips, the slip is attributed to an unforeseeable event, and the next plan is built on the very assumption that produced the first one. Scoring breaks that cycle, and it costs almost nothing, provided the record exists in the first place.

What makes later scoring possible is a small act of bookkeeping at the moment of commitment: write down the output you were forecasting, the threshold, the probability you assigned to clearing it, and the date by which you expect to know. Four lines. Captured at the decision they are a record; reconstructed afterwards they are a story, because memory adjusts a forecast towards whatever actually happened. Note the two or three drivers the ranking put at the top as well, since when a plan misses it is worth knowing whether it missed for the reason you expected or for one you never ranged. None of this requires a system; a dated page per decision is enough to begin.

This is a personal discipline before it is an institutional one. Anyone who records ten probability estimates and then checks them learns something uncomfortable and useful about their own range of error, most often that their confident calls were considerably less reliable than they felt and that their ranges were too narrow to start with. That is a finding about the forecaster rather than about the plans, and it is available to an individual without a programme, a policy or anybody else’s cooperation. Teams that build a habit of looking back usually acquire it from one or two people who started doing it privately, which is the realistic route into the wider habit described in building a risk-aware culture.

Incertive supports that habit with calibration tracking: a record comparing the probability estimates a user has made against the outcomes that user reports back, so a pattern of over-confidence or under-confidence becomes visible instead of remaining a suspicion. It is a mirror rather than an engine. The point is to let a plan owner see their own biases in their own numbers, and what they then choose to do about it, whether that means widening ranges, discounting their own certainty, or asking for a second opinion on a particular class of judgement, remains entirely theirs to decide.

Taken together the workflow is short enough to hold in your head and repeatable enough to run on every plan that crosses your desk, which matters a great deal more than any individual refinement inside it.

  1. Name the single quantity the decision turns on, together with the threshold it has to clear, in the units the commitment was actually made in.
  2. Find the handful of uncertain inputs that genuinely move that quantity, including the dependencies owned by other functions entirely.
  3. Characterise each one as a range, eliciting a minimum and a maximum before the most likely rather than accepting a single point.
  4. Propagate the ranges through the plan by simulation, so the output becomes a distribution of outcomes with frequencies attached.
  5. Rank the drivers by how far each one moves the outcome, and read that ranking as a shopping list for uncertainty reduction.
  6. Convert the distribution into a decision by comparing the probability of clearing the threshold against the confidence level you will fund.
  7. Record the probability, the threshold and the date you expect to know, so the forecast can be scored honestly once the outcome arrives.

The methods that do the quantifying

Uncertainty quantification is not one technique but a small family of them, and the practical skill lies in knowing which question each one answers. Three-point estimation gives you a defensible way to describe what you do not know about a single input. Monte Carlo propagation carries all of those ranges through the logic of your plan to produce a distribution of outcomes. Reference-class forecasting checks the resulting range against what actually happened to comparable efforts. Sensitivity analysis tells you which of the inputs is doing the work. None of them substitutes for another, and a serious analysis of a development plan usually ends up using all four, in roughly that order.

Three-point estimation and PERT

Ask an experienced engineer how long a work package will take and you will get a single number, delivered with a confidence that the number itself does not deserve. Ask the same person for the fastest it could credibly go, the duration they would bet on, and the worst case short of catastrophe, and you will get three numbers that are genuinely informative. That is the entire insight behind three-point estimation: the minimum, most likely and maximum are a format that human experts can actually answer, because each of the three corresponds to a concrete mental scenario rather than to an average they have never consciously computed and could not defend if challenged.

The format has a long institutional lineage. It entered mainstream practice through the Program Evaluation and Review Technique, developed for the scheduling of large defence programmes in the late 1950s, where planners needed some way to roll thousands of uncertain activity durations into a credible programme date. PERT’s contribution was to treat each activity as a range rather than a figure, and to compute an expected duration that leans heavily on the most likely value while still being pulled by the two extremes, so that an activity with a long tail of possible delay is never quietly recorded at its optimistic number. A PERT calculator now does that arithmetic in seconds.

The conceptual point most often missed is that three points are not themselves a distribution. They are constraints that imply one, and the choice of which shape to fit across them is a modelling decision with real consequences for the answer. The US Government Accountability Office’s cost estimating and assessment guide is explicit that a risk and uncertainty analysis can be conducted by a three-point estimate methodology, by a risk-driver approach that models the discrete risks themselves, or by a combination of both. The three-point route is the cheaper and far more common one, and for most development plans it is the right place to start.

Five shapes - triangular, lognormal, beta, uniform and normal are the probability distributions that cost estimators most commonly use.

Source: GAO-20-195G

Those shapes are not interchangeable, and each says something plain about the quantity being described. A triangular distribution expresses a bounded judgement with a clear most likely value, which is exactly what a three-point estimate is. A lognormal expresses a quantity whose downside is bounded at zero but whose upside has a long tail, which is how costs and durations usually behave: a task can overrun by three hundred percent far more easily than it can finish in negative time. A uniform says you genuinely know nothing except the bounds, a normal says variation is symmetric and arises from many small independent effects, and a beta offers bounded support with a smoother and more flexible interior. Matching the shape to the behaviour of the quantity matters far more than picking the most sophisticated one available, and reaching for a symmetric shape out of habit quietly asserts that your plan is as likely to come in early as late.

Monte Carlo propagation

Monte Carlo simulation is the general-purpose engine of uncertainty quantification, and its generality is the whole reason it dominates. It places essentially no restriction on the structure of the model it is applied to: the logic can contain thresholds, conditional branches, lookups, non-linear relationships and dependencies between inputs, and the method does not care, because it never needs to solve anything symbolically. The price of that freedom is computation rather than mathematical derivation. Where an analytical approach demands that you be clever, sampling demands only that you be patient, and modern hardware has made patience extremely cheap. Our guide to Monte Carlo simulation for business sets out the engine in full.

A single iteration is almost disappointingly simple. The simulator draws one value from each input distribution, evaluates the model once with that particular combination of values, and records the resulting outcome: one total cost, one completion date, one margin. That single result is one plausible version of the future and no more authoritative than any other. The power comes entirely from repetition. Run the same procedure thousands of times and the recorded outcomes accumulate into a shape whose features stop moving: the centre settles, the tails fill in, and the percentiles you read off the result stabilise to a tolerance far tighter than the uncertainty you are trying to describe in the first place.

It is worth being blunt about what the method does not do. Monte Carlo propagation creates no information: it takes the judgement encoded in the input ranges and the structure encoded in the model, and tells you what the two jointly imply, with a numerical precision that can be flattering. If the ranges are too narrow, the output distribution will be too narrow, and it will look exactly as authoritative as a well-calibrated one. The same caution applies to the relationships between inputs, because treating every variable as independent is convenient and frequently wrong: labour rates and schedule duration move together, as do demand and price, and a simulation that ignores the link understates the tails by assuming the bad cases never arrive at once. Any serious Monte Carlo tool has to let the analyst say that two things are connected, which is why Monte Carlo simulation in project management is a discipline of elicitation first and arithmetic second.

Reference-class forecasting as the outside view

Every estimate built by decomposing your own plan is an inside view. You list the work packages, you reason about each one from what you know of your team and your technology, and you assemble the pieces into a total. The method has an obvious appeal, because nobody knows the specifics of your plan better than you do, and a structural weakness that follows directly from that appeal: the decomposition can only ever contain the problems you have thought of. The outside view asks a cruder and much less comfortable question. What happened to the last hundred efforts that looked broadly like this one, and what exactly makes yours different?

The empirical answer is unsettling enough to have acquired a name. Bent Flyvbjerg’s Iron Law of Megaproject Management holds that megaprojects run over budget, over time, over and over again, with nine out of ten exceeding their budget. A planner working purely from the inside view cannot see that pattern, because each individual plan contains a specific and locally plausible reason why it will be the exception: a better contractor, a simpler scope, a lesson learned from the last one. The base rate is visible only from outside the plan, by looking at the population the plan belongs to rather than at the plan itself.

9 in 10 - megaprojects run over budget, a base rate that no single plan’s internal reasoning ever predicts for itself.

Source: Bent Flyvbjerg

What makes the method operational rather than merely chastening is that base rates differ sharply by class, and selecting the class is most of the analysis. The study by Flyvbjerg, Holm and Buhl of 258 transport infrastructure projects found an average cost overrun of roughly 28 percent, but the headline conceals a wide spread: roads came in at about 20 percent, bridges and tunnels at about 34 percent, and rail at about 45 percent. A rail promoter who borrowed the road figure would be optimistic by more than a factor of two. Defining a class tightly enough to be relevant and broadly enough to have members is the craft of reference-class forecasting.

28% - average cost overrun across 258 transport projects, ranging from about 20 percent for roads to about 45 percent for rail.

Source: Flyvbjerg, Holm and Buhl (2002)

The outside view is best understood as a way of calibrating the range rather than of replacing the model. It rarely tells you what your project will cost, because no reference class is ever quite your project, and a class chosen loosely enough to be large is usually chosen loosely enough to be irrelevant. What it does tell you is whether the distribution you have built is plausibly wide, and in almost every documented case the answer is that it is not. If your simulated worst case is a 12 percent overrun and the class you belong to averages 28 percent, the argument you owe the board is not about simulation settings but about why your plan sits outside its own history.

Sensitivity analysis and the tornado diagram

Once a distribution of outcomes exists, the next question is not how wide it is but what is making it wide. The standard method is deliberately simple: take one input, move it across its plausible range while holding every other input at its central value, record how far the output moves, and then repeat for every input in the model. The result is a list of swings, one per variable, which can be sorted from largest to smallest. Nothing about the procedure is statistically subtle, and that is a considerable virtue, because the output is immediately interpretable by people who will never willingly read a probability density.

Plotted as horizontal bars ordered by magnitude, that list becomes the familiar tornado chart: the widest bar at the top, narrowing downward, which is where the name comes from. Each bar shows how far the outcome travels when that one input travels across its range, and the asymmetry of a bar shows whether the variable hurts you more than it helps, which is frequently the case and just as frequently ignored. A tornado diagram is the fastest available answer to the question of where management attention belongs, and sensitivity analysis is the discipline that produces it; our longer treatment of the method works through how to read one properly.

Two properties, and only two, make a driver dominant. The first is the width of its own plausible range: a variable you are deeply unsure about has far more room to move the answer than one you know well. The second is leverage, meaning how much the output changes for a given change in that input. A dominant driver needs both. An input with enormous leverage but a range of plus or minus one percent will sit near the bottom of the chart, and so will an input you know almost nothing about if the outcome barely responds to it. The chart effectively multiplies ignorance by consequence and then ranks the product.

This is precisely the gap that a single contingency figure leaves open. The GAO guide notes that a three-point method yields a contingency amount but gives no indication of which risks are most likely to affect program cost, so a reserve can be perfectly well sized and still offer no guidance at all about what to manage. The ranking supplies that guidance, and each of the two properties has its own remedy: a driver that dominates because its range is wide is an argument for buying information, through a pilot, a test or a firm quote that narrows the band before money is committed, while a driver that dominates because its leverage is high is an argument for changing the plan itself, through a different sequence, a contractual cap or a staged commitment. Acting on the percentile without the ranking is funding a problem rather than reducing it.

Where analytical and surrogate methods fit

Sampling is not the only way to propagate uncertainty, and it is worth knowing the alternatives well enough to recognise when somebody is offering you one. For simple models, uncertainty can sometimes be carried through in closed form: if the output is a straightforward function of the inputs and the input distributions are well behaved, the distribution of the output can be derived algebraically rather than estimated by repetition. The appeal is exactness and speed, with no sampling error at all. The limitation is that real plans are rarely simple enough, because a single threshold, cap or conditional branch in the logic is usually sufficient to put the derivation out of reach.

A second family of methods comes from scientific computing, where the binding constraint is different. If one run of a model takes hours on a cluster, running it ten thousand times is not an option, so practitioners build a cheap statistical approximation of the expensive model, known as an emulator or a surrogate, and run the uncertainty analysis against the approximation instead. The National Academies study on assessing the reliability of complex models covers emulation and reduced-order modelling alongside verification, validation and prediction, which is a fair indication of how central the technique has become in that world.

2012 - National Academies study that places uncertainty quantification at the centre of both model validation and prediction.

Source: National Research Council

None of this is likely to trouble anyone planning a product launch or a capital programme, and it is better to say so plainly than to imply a sophistication the problem does not require. A business planning model is cheap to run: evaluating a cost build-up or a revenue forecast takes microseconds, so the computational argument for building a surrogate simply never arises. The reason to know these methods exist is mainly defensive, so that a vendor’s claim of an analytical shortcut can be judged on whether the underlying model is simple enough to deserve one. For a development plan, sampling is almost always the practical answer, and the effort saved belongs in the inputs.

Reading the output: percentiles, S-curves and confidence levels

A simulation produces thousands of outcomes, and the value of the whole exercise depends on reducing them to a handful of numbers that a decision-maker can act on without misreading them. This is where uncertainty quantification most often goes wrong in practice: not in the modelling but in the reporting, because the vocabulary of percentiles and confidence levels looks self-explanatory and is not. A P80 is a commitment with a specific and checkable meaning. An S-curve answers a question that most contingency arguments never get as far as asking. And a confidence level on cost alone says remarkably little about whether the plan as a whole will hold.

What P10, P50 and P90 actually commit you to

The definitions are mechanical. If a simulation of your programme produces ten thousand outcomes and you sort them from best to worst, the P90 cost is the value at which ninety percent of the simulated outcomes fall at or below: nine in ten of the futures the model considers come in at that number or better. P50 is the median, with half the outcomes on either side of it. P10 is the optimistic end, beaten in only one simulated future out of ten. The arithmetic is trivial and uncontroversial, which is exactly why the interpretation deserves the care that the arithmetic does not.

Committing to a P50 date means accepting a coin flip, and saying it in those words changes conversations. A great many organisations publish a plan that is, on their own analysis, roughly as likely to be missed as met, and then treat each individual miss as a failure of execution rather than as the predictable consequence of the confidence level they chose. There is nothing wrong with a P50 as a target, since a target that is comfortable to hit generates slack and invites it to be consumed. There is a great deal wrong with presenting a P50 as a promise, because the promise carries an implied likelihood that the number does not support.

The distances between percentiles carry the information that matters most, and they are the part of the output that gets quoted least. The gap between P50 and P80 is the price of confidence: it is what the organisation must fund, in money or in calendar time, to move from an even chance to a comfortable one. The gap between P10 and P90 is a measure of how much you actually know, and it is the one figure in the whole output that resists spin, because a very wide band is an admission and a very narrow one is a claim that has to be defended. Reading the full probability distribution rather than a single summary keeps both facts in view.

The most common and most expensive confusion in this area is between a target and a commitment, and the resolution is to use two numbers deliberately rather than one ambiguously. A team can reasonably work to a P50 internally while the organisation commits externally to a P80, holding the difference as contingency it has sized rather than guessed. The failure mode is publishing a single number for both purposes, at which point either the team is working to a date it cannot hit or the organisation is sitting on slack it cannot see. Our guide to probabilistic forecasting works through how to hold the two levels apart without the internal figure quietly becoming the external one.

The S-curve and the contingency conversation

The S-curve is the same distribution read cumulatively. Instead of showing how many simulated outcomes landed near each value, it shows, for every value along the horizontal axis, the probability that the final result comes in at or below that value. The shape that gives the curve its name follows directly: flat at the optimistic end where few outcomes live, steep through the middle where most of the probability sits, flattening again into the expensive tail. Its practical virtue is that it answers the question a budget holder actually has, which is not what the cost will be but what number they would have to fund in order to be comfortable.

The GAO guide works an example worth quoting precisely, because the figures state something that estimators rarely see written down so clearly. For the air vehicle in that example, the point estimate is $2.47 billion. The cost at the 50 percent confidence level is $2.61 billion. Budgeting to an 80 percent confidence level raises the figure to $2.79 billion, which is about $320 million above the point estimate. Three numbers read off a single S-curve convert contingency from a negotiation into a calculation: the reserve is simply the distance between the point estimate and whichever confidence level the organisation has decided to fund.

$2.79 billion - the air vehicle cost at an 80 percent confidence level, about $320 million above the $2.47 billion point estimate.

Source: GAO-20-195G

The structural lesson, though, sits in the first two figures rather than the third. The point estimate of $2.47 billion fell below even the 50 percent confidence level of $2.61 billion, which means the estimate as originally presented had less than an even chance of being achieved before anybody argued about contingency at all. That is the normal condition of point estimates rather than an anomaly, because a build-up of most likely values accumulates the optimistic side of every component and leaves the correlations and the tails out of the total. The gap is also entirely invisible without an uncertainty analysis, which is why it survives so comfortably inside capital project budgets.

$2.61 billion - the same programme cost at a 50 percent confidence level, above a point estimate that therefore had less than an even chance of holding.

Source: GAO-20-195G

The guide is equally firm about documentation, asking that the probability distributions used and the resulting S curve be fully documented. The reason is institutional rather than academic. A confidence level is only defensible if a reviewer can see which ranges produced it, and an undocumented S-curve is indistinguishable from a decorated guess six months later, once the people who built it have moved on to other programmes. Any tool that reports percentiles should therefore report the assumptions behind them as well: Incertive shows the distribution and the S-curve with P10, P50 and P80 marked, and publishes its methodology, including its limitations.

Joint confidence level: cost and schedule together

Cost and schedule are not separate bets, and reporting a confidence level on each in isolation hides the thing a sponsor most needs to know. A plan can sit at 70 percent on cost and 70 percent on schedule and be well below 70 percent on both together, because the two are driven by overlapping causes: the labour shortage that stretches the programme also inflates its cost, so the futures in which the budget holds are disproportionately the ones in which the date holds, and the futures in which either fails tend to be the ones in which both do. What a sponsor is promised is the joint event, and the probability of a joint event is always the smaller number.

Serious programme offices therefore measure a joint confidence level, a single probability covering cost and schedule at once. NASA’s interim directive on programme and project management requires that, for projects and single-project programmes with a life-cycle cost of $1 billion or more, Mission Directorates plan and budget programmes and projects based on a 70 percent JCL or as approved by the Decision Authority. The JCL is informed by probabilistic analysis of development cost and schedule duration, and any justification for budgeting below the 70 percent confidence level must be included in the KDP-B Decision Memorandum.

70% - the joint confidence level on cost and schedule that NASA uses as the budgeting basis for projects of $1 billion or more.

Source: NASA NID 7120.122

The lesson for commercial teams has very little to do with spaceflight. A public agency has written its confidence level into policy, named the document in which a departure from it has to be justified, and specified that the number comes from a probabilistic analysis of cost and duration rather than from a negotiation between sponsors. Most companies do not come close to that standard: they commit to dates and budgets without ever stating a confidence level, which makes a shortfall impossible to distinguish from bad luck and impossible to learn from, because there was never a stated probability against which the outcome could be compared. An organisation that simply records the confidence level behind each commitment has, at no cost, created a record it can audit.

Coverage factors and the vocabulary of expanded uncertainty

One field settled this vocabulary decades ago and settled it precisely. Metrology, the science of measurement, cannot report a result at all without reporting how much it does not know, so it developed a careful language for doing exactly that. NIST Technical Note 1297 distinguishes Type A evaluation of standard uncertainty, derived by the statistical analysis of repeated observations, from Type B evaluation, derived by judgement from whatever other information is available: instrument specifications, calibration certificates, prior experience, published data. The distinction is not a ranking of quality. Both are legitimate components of a stated uncertainty, and the honest move is to label which component is which.

The components are then combined into a combined standard uncertainty, and because a standard uncertainty on its own describes a fairly narrow interval, it is multiplied by a coverage factor to give an expanded uncertainty: the interval within which the value is expected to lie at a stated level of confidence. A coverage factor of k equal to 2 defines an interval with a level of confidence of approximately 95 percent. Every quantity in that chain has a name, a definition and a convention for how it is reported, so that two laboratories on different continents can read each other’s numbers without first having to negotiate what the numbers mean.

k = 2 - the coverage factor that defines an interval with a level of confidence of approximately 95 percent.

Source: NIST TN 1297

Set that against the way a development plan is usually reported and the contrast is uncomfortable. Business planning has had access to this vocabulary for decades and has mostly declined to borrow it: budgets arrive as single numbers with no stated interval, no coverage factor, no indication of whether the range behind them came from data or from judgement, and no label distinguishing the two. NIST’s own framing makes the point more sharply than any argument about method can, holding that the value of a measured result is largely determined by the quality of the uncertainty statement that accompanies it. A development plan is a prediction rather than a measurement, so its uncertainty is larger and the case for stating it is stronger still. The practical ask is modest: alongside every committed figure, a range, a confidence level, and one line on where the range came from.

point estimate50%80%0%contingency to reach 80%Total costConfidence
The S-curve turns contingency from a negotiation into a measurement: the gap between the point estimate and the confidence level you intend to fund is the number to budget (illustrative).

Correlation: the mistake that makes every plan look safer than it is

Of all the ways a quantified plan can mislead the people who rely on it, the most common is also the least visible. A team gathers honest ranges for every uncertain input, runs the simulation, reads a reassuringly tight distribution, and funds a contingency sized to its P90. Nothing in that sequence looks wrong. The flaw sits in an assumption nobody made out loud: that the inputs vary independently of one another, each wandering inside its own range without reference to what the others are doing. In most development plans that assumption is false, and when it is false the model does not fail loudly. It quietly produces a plan that looks safer than it is.

Why independence is the wrong default

Independence is not usually a conclusion anybody reached. It is what a spreadsheet does when it is handed a list of ranges and asked to sample from them, and it is what most simulation tools do unless they are told otherwise. The consequence is easy to picture. Every iteration draws a value for each input without reference to the values drawn for the others, so a large number of iterations pair a high figure on one line with a low figure on another. The expensive labour month arrives in the same scenario as the cheap steel delivery; the regulator takes an extra quarter in the same scenario in which integration testing finishes early. None of those scenarios looks absurd on its own, and that is precisely what makes them harmful.

What they do is offset. A scenario in which one input overruns while another underruns lands near the middle of the output range, because the damage on one line is paid for by the saving on another. Extreme totals, by contrast, require a majority of the inputs to go wrong at the same time, and an independence assumption treats that conjunction as a coincidence of coincidences. Multiply enough small probabilities together and the model concludes that broad, simultaneous bad news effectively does not happen. The simulated extremes thin out, the body of the distribution thickens, and the plan acquires a statistical stability that the real programme never had.

The error has a direction, and it is always the same one. Treating dependent inputs as independent does not blur the answer symmetrically; it compresses the distribution towards its centre. The mean and the median barely move, which is why nothing looks wrong on the summary slide, but the spread shrinks, the P90 sits closer to the P50 than it should, and the contingency sized from that P90 is too small. A planner reading the output is told that the plan is more predictable than it is, and the figure most likely to be quoted to a board - the number the organisation claims it is very unlikely to exceed - is the figure most badly affected.

79% - was the average cost overrun across more than 500 major projects reviewed by McKinsey, a consistency that is hard to explain as a collection of unrelated, independent mistakes.

Source: McKinsey

Independence is occasionally a fair description of reality. The failure rate of a bought-in component and the time it takes to recruit a country manager really may have nothing in common underneath them. The problem is not that independence is always wrong; it is that it is applied silently, as a property of the software rather than a judgement about the plan. Anyone collecting three-point estimates has already thought hard about each line in isolation, and the method set out in our guide to Monte Carlo simulation is only ever as good as the relationships it has been told about. The question to put to every pair of large inputs is short: if this one came in badly, would that one be more likely to come in badly too?

Shared drivers and the problem of a common shock

Correlation in a development plan is very rarely a statistical curiosity. It is almost always a causal fact that somebody on the team could state in a sentence, because two quantities move together for the obvious reason that something underneath moves them both. Labour rates and material prices are the clearest case. They are separate lines in the cost model and separate conversations with separate suppliers, but both respond to the same monetary and industrial conditions, and a year in which concrete, steel and freight all cost more is as a rule also a year in which trades are scarce and wages rise. Modelling them as two unrelated ranges invites the simulation to imagine a world of expensive materials and cheap labour, which is not a world that has often existed.

Schedule and cost are joined in the same way, and the link is mechanical rather than statistical. A build that runs four months long keeps a management team, a site, a financing facility and a set of rented machines in service for four extra months, so duration and total spend are two readings of a single underlying event. Yet many plans carry an independent range for the schedule and an independent range for the budget, and then report a cost P90 drawn largely from scenarios in which the programme is late but somehow not expensive. That flatters the plan twice over, once in the cost tail and once in the schedule tail, and the error compounds wherever time-related overhead is a large share of the total. Capital projects are where it shows up most starkly.

The commercial side of a development plan has the same structure. A launch model typically carries a cost of customer acquisition and a conversion rate as two separate assumptions, each with its own range, when in practice both are reading the same thing: how much genuine demand exists beyond the first enthusiasts. Push past the audience that was already looking for the product and acquisition gets more expensive while conversion gets worse, for one reason rather than two. A model that lets the two vary independently will generate a comfortable number of scenarios in which traffic is cheap and converts beautifully, and those are exactly the scenarios that make an ambitious revenue forecast look achievable.

Concentration of resources and suppliers produces the same effect from a different direction. Three workstreams that each depend on the same scarce specialist team do not have three independent schedule risks; they have one risk expressed three times, and the day that team is pulled onto something urgent all three slip together. Several suppliers sourcing through the same shipping lane, or built around the same constrained component, are similarly one exposure wearing different names. The useful consequence of all this is that dependency does not usually have to be measured. It has to be noticed. The team already knows which lines share a driver, and that knowledge is available to a leader with no statistical training long before any data exists to estimate it from.

How understated dependency narrows the distribution

It is worth being precise about where the distortion lands, because it is not spread evenly across the output. Around the middle of the distribution the independence assumption does comparatively little harm: the central scenarios involve a mixture of mild good news and mild bad news whatever the dependency structure, so the P50 is reasonably robust. The damage accumulates in the tails, and above all in the upper tail of cost and duration, because that is the region which only exists when many things go wrong at once. Dependency is the mechanism that makes simultaneous bad news plausible. Remove it and the model has no way to manufacture the correlated pile-up that produces a genuinely bad year.

That matters because the upper tail is not a statistical ornament. It is where the outcomes that threaten the organisation live: the overrun that exhausts the facility, the delay that pushes a launch past a contractual date, the cost that turns a positive business case negative. Everything the quantification exercise is actually for - sizing a contingency, choosing a confidence level to fund, deciding whether the downside is survivable - is a question about the part of the distribution that an independence assumption quietly deletes. A plan can be modelled with real care at the P50 and be completely unprepared at the P95.

Fat tails in large programmes are also an empirical finding rather than a theoretical worry. In a study of roughly 1,500 IT projects, Flyvbjerg and Budzier reported an average cost overrun of around 27 percent, but the striking result was the shape of the distribution rather than its average: a substantial minority of projects did not overrun slightly, they overran catastrophically. That work is the most direct available rebuttal to a model whose extremes have been averaged away, because no simulation built on independent inputs will generate events of that magnitude at anything like the rate at which the real world generates them.

One in six - of roughly 1,500 IT projects studied was a black swan, with a cost overrun of about 200 percent and a schedule overrun of about 70 percent.

Source: Harvard Business Review

The practical symptom is specific enough to be diagnostic. If an organisation keeps producing P90 figures that reality beats, the first instinct is to widen every input range, and that is usually the wrong correction. Ranges that were individually honest do not become dishonest in aggregate by accident; what has gone missing is the joint behaviour. A quick test is to take the last several completed plans, find where each actual outcome fell on the distribution published at approval, and look at the pattern. Outcomes clustering above the stated P90 across multiple unrelated plans is not bad luck repeated, it is a structural property of how those models were built, and comparing scenarios side by side is often the fastest way to expose it.

Handling dependency without a statistics team

The encouraging part is that handling dependency well does not require a quantitative analyst, and the attempt to do it precisely is often what stops teams doing it at all. A correlation coefficient looks rigorous and is almost never defensible: nobody in the room can explain why the figure should be one value rather than a slightly different one, so the discussion stalls and the assumption quietly reverts to none at all, which is the single setting everybody can be certain is wrong. A coarse, honest judgement beats a precise, invented one. Three buckets are usually enough: these two move together strongly, these two move together a little, these two are unrelated. Each of those statements is something a team can argue about on the evidence of the plan itself.

Better still, restructure the model so that the dependency never has to be specified. If two quantities move together because a third thing moves them both, the honest model contains that third thing. Make input-price conditions a single uncertain quantity and derive the labour and materials lines from it, and the relationship between them appears on its own, in the right direction and at roughly the right strength, without anybody naming a number for it. Do the same with time: model the schedule as uncertain and let time-related overhead follow from it rather than treating the two as separate guesses. The shared driver is modelled once, as its own uncertainty, and everything that depends on it inherits the consequence. That is usually cleaner to build, easier to explain and far easier for a reviewer to challenge.

Then stress the result deliberately rather than relying on the simulation to find the bad cases on its own. Take the three or four inputs that most affect the outcome, set each of them at its pessimistic end at the same time, and ask whether the plan survives. This is a scenario question rather than a statistical one, and it answers something a distribution alone cannot: not how likely the correlated pile-up is, but whether the organisation could absorb it if it arrived. If the answer is that it could not, the right response is a structural change to the plan, such as a phased commitment, a second supplier or a decision point placed before the irreversible spend, rather than a larger contingency line. Teams routinely find that this one exercise changes more about a plan than the simulation did.

Finally, write the assumption down. The GAO Cost Estimating and Assessment Guide asks that documentation fully describe the probability distributions used and the resulting S curve, and the reason is not bureaucratic tidiness. A dependency assumption that exists only in the modeller’s head cannot be reviewed, cannot be inherited by whoever takes the plan over, and cannot be defended at a gate when a sceptical reviewer asks why the cost tail is as thin as it is. Writing it down also exposes the cases where no justification exists. A sentence such as “labour and materials are treated as moving together because both follow input-price conditions” survives scrutiny; the sentence nobody wants to write is that every input was assumed to go its own way because that was what the tool did by default.

Uncertainty quantification across the development lifecycle

Uncertainty quantification is usually sold as an event: a study commissioned ahead of a funding decision, delivered as a deck, and filed. Treated that way it is worth something, but much less than it should be. The useful version is a thread that runs the length of the plan, asking the same question at every stage and expecting a different answer each time, because the stock of knowledge behind the plan keeps changing. What follows is what the discipline is for at each stage of a development effort, what good practice looks like there, and what the characteristic failure is.

Concept and feasibility: wide ranges, honestly stated

At the earliest stage almost nothing is irreducible. Very little of the uncertainty around a concept is the random variation that will remain however much work is done; nearly all of it is simply ignorance, the epistemic kind that more information can remove. Market size, technical feasibility, the cost of the hard part, whether the regulator will take a view at all: these are not unknowable, they are unknown, and the ranges that describe them are necessarily wide. A cost range at concept stage that spans a factor of two is not a failure of rigour. It is an accurate report of how much has not yet been found out.

The temptation at this point is to narrow the range on paper, because a wide range reads as a weak case and a tight number reads as competence. Teams presenting to an investment committee learn quickly that the confident figure gets the funding. This is the exact moment at which overruns are created. Nothing about the plan changes when the range is compressed; only the organisation’s understanding of it changes, and it changes in the direction of unpreparedness. A commitment made against a narrowed range is a commitment made without the contingency, the decision points and the fallback that the real width would have demanded. The width was the warning, and it has been edited out.

It helps to be clear about what quantification is for this early, because it is not prediction. Nobody can forecast the cost of a programme whose scope is still moving. The purpose is to decide two things: whether the opportunity is worth the cost of finding out more, and which unknown to attack first. A distribution that spans an uncomfortable range answers both. If even the optimistic end fails to clear the hurdle, the answer is no and the money is saved before it is spent. If the pessimistic end is survivable and the optimistic end is attractive, the answer is to proceed to the next stage and spend on information, and the ranked list of what drives the spread says what to spend it on.

About 20% - of new business establishments fail within their first year and about half are gone by year five, which is the base rate any new venture plan is arguing against.

Source: US Bureau of Labor Statistics

Design and planning: buying information to narrow the range

Between concept and approval the range should narrow. The question that separates a disciplined programme from an optimistic one is why it narrowed. A range that tightens because the team bought information has become more accurate. A range that tightens because the submission date got closer, or because the previous version was uncomfortable to present, has become less accurate while looking more professional. The two are indistinguishable in the output and entirely different in consequence, which is why the provenance of every narrowing is worth recording: what did we learn, and which number did it change?

Treated properly, this stage is a series of purchases. The ranked driver list says where the uncertainty is concentrated, and each item on it has a price attached to resolving it. A prototype settles whether the hard technical step works. A soil survey replaces an assumption about ground conditions with a measurement. A pilot in one region replaces a guess about demand with an observation. A supplier quotation, a letter of intent, a technical spike, a pre-application meeting with a regulator: each converts a band of epistemic uncertainty into something much closer to a fact, at a known cost and over a known period. The discipline is to buy in the order the driver list dictates rather than in the order that is most comfortable.

The test to apply to each purchase is whether it will move the probability of success enough to justify its price, and that test is only available once the probability exists. Without a quantified baseline, a decision to spend on a pilot is an argument about diligence; with one, it is a comparison between the cost of the pilot and the amount of spread it can be expected to remove from the thing that matters most. The same logic applies to the shape of the plan itself. Incertive generates and compares plan variants, a full launch against a phased rollout for instance, so a team can see which structure best matches the risk it is willing to carry rather than arguing about that in the abstract.

The approval gate: where the probability becomes the decision

The gate review is where the method earns its keep, because it changes the thing being debated. A gate built on a point estimate becomes an argument about optimism. The sponsor presents a number, the reviewers suspect it is too low, and neither side can do much more than assert, so the discussion settles on tone: does the team seem to have thought about it hard enough? A gate built on a distribution becomes a decision about a number. What is the probability of coming in under the threshold? At what confidence level are we funding this? What would have to be true for that probability to become acceptable? Those are answerable questions, and they are answerable in front of the people who will have to live with the answer.

The most common correct answer at a gate is neither yes nor no. It is: proceed once named actions have moved the odds. A plan whose probability of clearing the threshold is mediocre is rarely a plan that should be killed; it is usually a plan with two or three dominant uncertainties that have not yet been resolved, and the right decision is to authorise the work that resolves them and re-test before the irreversible commitment. That is why a go, no-go or conditional recommendation is more useful than a binary verdict, and why the structure of a go or no-go decision deserves as much design attention as the analysis feeding it. A conditional approval with named conditions and a date is a decision; an approval with a caveat in the minutes is not.

The idea that an organisation should state the confidence level it funds to is not a theoretical nicety. NASA requires projects and single-project programmes with a life-cycle cost of a billion dollars or more to produce a joint cost and schedule confidence level informed by probabilistic analysis of development cost and schedule duration, and directs that Mission Directorates plan and budget those programmes on the basis of a 70 percent joint confidence level, or as approved by the Decision Authority. Where a programme is budgeted below that level, the justification has to be included in the KDP-B Decision Memorandum. The lesson for a commercial organisation is not the specific figure. It is that the confidence level is a policy choice, that it can be set in advance, and that departing from it should cost somebody a written explanation.

70% - is the joint cost and schedule confidence level on which NASA Mission Directorates plan and budget projects of a billion dollars or more, or as approved by the Decision Authority.

Source: NASA NID 7120.122

Execution: re-running the question as the facts change

A quantification performed once at approval and never revisited is a photograph of a world that has since moved on. Within weeks of a start, things are known that were guesses at the gate: the first supplier contract is signed at a real price, the long-lead item has a confirmed delivery date, the first increment of the build has an actual productivity rate attached to it. Each of those facts collapses a range to a point, and the correct consequence is a tighter distribution and a smaller required provision. Re-running the analysis is how that benefit is realised. A programme that never re-runs keeps carrying money and float against uncertainties it has already resolved, which is its own quiet form of waste.

The more valuable signal is the opposite case. If a plan is a third of the way through and the distribution has not tightened, something is wrong that the progress report will not show. Either resolved unknowns are being replaced by new ones at the same rate, which says the scope is still moving, or the early ranges were narrower than the knowledge justified and reality is now filling in the width that was edited out. Both are discoverable months before they surface in a variance report, and both are invisible to a tracking system that compares spend against a fixed baseline rather than comparing today’s distribution against the one published at approval.

All of this depends on a shifted distribution being treated as information rather than as a confession. The organisational instinct is to read a re-baseline as an admission of failure, and the predictable result is that teams stop updating: the model is quietly frozen at the figure that was approved, and the quantification degrades into a compliance artefact. A distribution that has moved is telling the organisation something it needs, early, while options still exist and while the cost of acting on it is low. Re-baselining honestly is therefore a cultural problem more than a methodological one, and it is solved by making an accurate update cheap to give and safe to receive.

Closeout: scoring the forecast you made

The last step is the one almost every organisation skips. When a project closes, the file records what it cost and when it finished. It almost never records what the approved distribution said the chances were, or whether the outcome was consistent with it. Without that comparison the forecasting process has no feedback at all: the same people make the same kind of estimate next quarter with no evidence about whether their last ten were biased, and the organisation accumulates experience without accumulating skill. Closing the loop costs very little. It requires that the probability stated at the gate be written down in a form that can be checked later, and that somebody actually check it.

The comparison has to be made across many decisions rather than one, because a single outcome can neither validate nor refute a probability. Suppose a plan is given a seventy percent chance of landing under budget (illustrative) and then lands over it. Nothing has been shown about the quality of the assessment; three plans in ten were always supposed to do exactly that. What is informative is the pattern. If the decisions an organisation calls eighty percent likely come in about eighty percent of the time (illustrative), its probabilities mean something and can be used to allocate capital. If they come in half the time, the numbers are decoration, and the right response is not to abandon probability but to correct the estimating that produced it.

That is the purpose of calibration tracking: a record comparing the probability estimates a user made against the outcomes they report back, so the person can see where their own judgement runs hot or cold. It is deliberately personal and deliberately unglamorous. Nothing in the exercise is harder than keeping the record, and the value lies entirely in what the person does with it, because the bias being corrected is the forecaster’s own and only the forecaster can correct it. An organisation that keeps that record for a year has something no amount of modelling sophistication can substitute for: evidence about whether its own stated probabilities can be believed, which is the foundation of a risk-aware culture and the whole reason the loop is worth closing.

Integration effortSpecialist availabilityApproval timingSupplier pricingAdoption rateScope changesbetter than planworse than planSwing in the outcome
Ranking the drivers converts a distribution into a plan of action: the inputs at the top are where buying information or adding protection changes the odds most (illustrative).

What to look for in software that quantifies uncertainty

The category is crowded and the labels are unreliable. Products sold as scenario planners, risk platforms, forecasting suites and decision engines overlap almost completely in their marketing and diverge sharply in what they actually compute, and the word uncertainty appears somewhere on nearly all of them. Rather than trying to classify vendors by their own vocabulary, it is more productive to carry a short list of capability questions into every trial and answer them yourself with your own plan. Each of the six below can be settled in an afternoon, and each separates a tool that will change how a development decision gets made from a tool that will produce a tidier version of the spreadsheet you already have. Comparison guides to risk modelling software and Monte Carlo simulation software are a reasonable way to assemble a shortlist, but the questions matter far more than the list does.

Does it propagate uncertainty, or only compare cases?

This is the single most important distinction in the category, and it is the one the marketing most reliably obscures. A large share of products sold for scenario work are case comparison tools: they let you duplicate a model, change a handful of assumptions, and line the copies up side by side as a base case, an upside and a downside. That is a genuinely useful discipline and it is not uncertainty quantification. Three cases produce three points. They carry no information about how likely any of them is, which means the reader still has to supply the only judgement that actually matters, namely how much weight to put on the bad column.

Propagation works differently. Every uncertain input carries a range rather than a value, the ranges are sampled together across many thousands of trials, and the arithmetic of the plan is run once per trial, so the output is a population of outcomes rather than a handful of hand-picked ones. From that population you can read a likelihood for any threshold you care about: the share of trials that finish inside the approved budget, that land before the committed date, that clear the margin the board was promised. The question changes from what happens if the integration is late to how often, across everything that could plausibly happen, the integration being late is what costs us the date.

The twenty minute test is to take one real threshold out of your own plan and ask the tool for the probability of clearing it. If the answer is a percentage the tool derived from the ranges you entered, it propagates. If the answer is a column you have to choose between, or a single figure labelled contingency, it compares cases. Two secondary tells are quick to check: look for percentile outputs such as P10, P50 and P80, and look for a cumulative curve rather than only a bar chart of three scenarios. Tools built on spreadsheet duplication rarely have either, for reasons set out in our note on the limits of forecasting in spreadsheets and in the comparison of what-if analysis software.

None of this makes case comparison worthless, and a buyer should resist the temptation to treat it as a disqualification. Discrete variants are the right structure when the choice itself is discrete, because a full launch and a phased rollout are different plans rather than different draws from one plan, and each deserves a distribution of its own. The mistake is accepting a set of cases in place of a probability and then briefing the cases as though a probability had been computed behind them. A tool that does the job properly will give you both: a named variant and, for each one, a probability of success stating what share of simulated scenarios met the goals you set.

Does it rank the drivers?

A tool that returns only a distribution has told you the size of your exposure and nothing about what to do with it. Learning that a programme has roughly a thirty-five percent chance of landing inside its budget (illustrative) is a better starting point than a single point estimate, but it does not tell anyone which meeting to call on Monday morning. The useful output is a ranked and quantified view of which inputs actually move the outcome, because that is the list which converts an analysis into work: the top item is where you buy information, renegotiate, prototype or add float, and the items at the bottom are where you stop spending estimating effort altogether.

No indication - of which risks most affect program cost comes out of a three-point estimate methodology, which yields a contingency amount and leaves the driver question entirely unanswered, according to the federal cost estimating guide.

Source: US Government Accountability Office, GAO-20-195G

The capability shows up as a ranking with magnitudes attached, usually drawn as a tornado diagram, in which each bar is the swing in the outcome attributable to one input moving across its own range. A tool that lacks the capability will hand you a distribution, a contingency figure, or a correlation table you are expected to interpret for yourself. The practical difference is large. A ranked view with units on it is actionable by somebody who has never opened a statistics textbook, whereas a correlation matrix is an invitation to argue about methodology. Our primer on sensitivity analysis sets out why the ranking is usually more decision-relevant than the distribution it was derived from.

To test it, run your plan and then ask one question: which three inputs should we go and reduce first, and how much would reducing each of them move the answer? A tool with the capability answers immediately and in the units of your plan, in weeks or in currency. A tool without it either cannot answer at all or asks you to re-run the model by hand with one input frozen at a time, which is precisely the manual labour the software was supposed to remove. Be sceptical of rankings presented without magnitudes, because an order with no distances in it tells you very little about where the next pound of effort should go.

Does it handle dependency between inputs?

Independence is the quiet default in most quantified plans, and it is usually wrong. The inputs in a development plan are not strangers to one another. Labour rates and material costs are driven by the same inflation. A schedule slip and a cost increase are frequently the same event seen from two angles. The assumptions about adoption, pricing and churn in a launch plan all depend on whether the market turns out to want the product at all. When a tool samples those inputs independently, a bad draw on one is routinely offset by a good draw on another, the offsetting cases crowd into the middle of the output, and the tail where several things go wrong together comes out thinner than reality.

Understating the tail is not a small error, because the tail is the part of the distribution the whole exercise exists to find. A plan whose P50 is roughly right and whose P90 is quietly optimistic will pass review, get funded at a confidence level nobody can actually honour, and then fail in exactly the correlated way the model excluded by construction. The remedy does not require a statistician. There are two serviceable moves: state directly that two inputs move together, or restructure the plan so the shared driver is modelled once as its own uncertain input and then flows into everything it affects. The second is usually the more honest, because it makes the mechanism visible instead of burying it inside a coefficient somebody guessed.

The check is a short conversation followed by a single comparison. Describe two inputs in your own plan that are driven by the same underlying thing and ask how the tool represents that relationship; an answer which amounts to inputs are treated as independent is still an answer, and you can price it in once you know. Then look at the upper percentiles with and without the dependency expressed and see whether they move. If the output is identical either way, the dependency was never in the model. Guides to risk analysis software rarely make this a headline feature, which is exactly why a buyer has to raise it directly rather than wait to find it on a comparison table.

Can the person who owns the decision use it?

The historical failure of probabilistic tooling has very little to do with mathematics. The methods have been settled for decades; the problem was that using them required a modeller. Somebody had to build the model, choose the distributions, wire up the dependencies and interpret the output, which put a specialist between the decision and the analysis. Two consequences followed, and both are worse than they sound. The analysis happened once, at the gate, because each run cost a specialist a week of effort. And the person who owned the plan did not own the analysis, so whenever the numbers were inconvenient they were treated as the analyst’s opinion rather than as a property of that person’s own plan.

A development plan is not a static object. Assumptions resolve weekly: a supplier confirms a lead time, a regulator asks for another study, a senior hire falls through, a pilot comes back better than anyone hoped. Quantification is only management information if it can be refreshed at the speed those facts arrive, which means the cost of the second run matters far more than the cost of the first. This is the practical argument for tools that take a description of a plan rather than a built model. Incertive, for instance, accepts a plain-language description of a plan and identifies the uncertainties and risk factors inside it, which is what makes re-running after a change a routine act rather than a small project.

Test this by handing the trial to the wrong person on purpose. Let the person who owns the plan, not the analyst who kindly volunteered to help, produce the first result unaided, and watch where they stall and what they have to ask. Then change one assumption and time the second run. Seconds is the standard; if it takes days you have bought a study rather than an instrument, and the study will be out of date before the next steering meeting. Ask too what happens when that one capable person leaves, because a model only a single employee can operate is a point of failure dressed up as a capability.

Does it leave a record you can be held to?

A forecast is a claim about the future, and a claim nobody wrote down cannot be shown to have been wrong, which is precisely the problem. Organisations that quantify uncertainty for years without getting any better at it almost always share one trait: no durable record of what was actually forecast. The numbers lived in a deck, the deck was superseded, and when the programme landed four months late nobody could say whether the model had warned them or not. Without the record there is no feedback loop, and without a feedback loop an estimating culture cannot calibrate, so the same optimism is re-applied to the next plan with undiminished conviction.

The capability to look for here is narrow and concrete. A tool should preserve what was forecast, against which threshold, on what date, together with the inputs and ranges that produced it, so that the forecast can be compared with the outcome later without anyone reconstructing it from memory or from an email thread. That record is also what makes a difficult review survivable in the other direction. When a plan is challenged after the fact, being able to show the ranges as they stood at approval, and that the eventual outcome fell inside them, is a far stronger position than arguing from recollection about what was known at the time.

The test takes two minutes and very little passes it by accident. Run an analysis, change several inputs, and then try to retrieve the earlier version: can you still see what it assumed, when it was produced and what probability it reported? If the earlier result has been silently overwritten, you are evaluating a calculator rather than a record. The weaker but acceptable form of this capability is an export you file yourself; the strong form keeps the comparison for you, so that a tendency to forecast too narrowly becomes visible across a dozen plans instead of being rediscovered on each one. Either way the discipline belongs to the organisation rather than the vendor, as our guide to probabilistic forecasting argues at greater length.

Does it state its own limitations?

A tool that presents itself as producing certainty is describing something that cannot exist, and the claim should be read as information about the vendor rather than about the product. Every quantified plan rests on inputs that were elicited from people, a structure somebody chose, and a set of uncertainties nobody thought to include. None of that is a defect to be apologised for; it is the condition of the work. What distinguishes a serious tool is that it says so plainly, names the assumptions it is making on your behalf, and tells you what its output does not cover before you have to find out for yourself.

The engineering disciplines that have been quantifying uncertainty for longest are the most explicit about this. Validation standards in computational modelling do not certify a model as correct. They quantify the degree of accuracy inferred from comparing a solution with data at a specified validation point, and they are candid that accuracy elsewhere in the domain remains a matter of engineering judgement. The National Research Council study of verification, validation and uncertainty quantification in complex models draws the same line in a different place: verification asks how well a model solves its own equations, validation asks how well it represents the real system, and the two questions have genuinely different answers.

Not pass or fail - is how the leading validation standard frames its own work: validation assesses model error at a specified validation point rather than issuing a verdict, and accuracy elsewhere stays a matter of engineering judgement.

Source: ASME V&V 20-2009

In software for development plans the equivalent of that candour is a published method. Prefer a vendor who will tell you, before you buy anything, how the output is produced in general terms, what it assumes, where it is weak and what it should not be used for; Incertive publishes its methodology, including its limitations. Treat the absence of such a page as a finding in itself. The twenty minute version of this test is to ask the vendor for three situations in which their tool gives a misleading answer. A good answer exists and is short. An answer that there are none tells you the vendor has either not looked or will not say, and both are reasons to keep the trial brief.

  • Does the output include a probability for a threshold we actually care about, or only a set of cases we have to choose between?
  • Can the tool tell us, with magnitudes in the units of our plan, which three inputs most move the outcome?
  • Can we express that two inputs move together, or restructure the plan so a shared driver is modelled once?
  • Can the person who owns the plan produce a result unaided, and re-run it in seconds after one assumption changes?
  • Does the tool preserve what was forecast, against which threshold and on what date, so it can be checked later?
  • Does the vendor publish its method and name the situations in which that method gives a misleading answer?

Seven ways uncertainty quantification goes wrong

Ranges anchored on the plan

The most common failure is also the hardest to see, because the output looks exactly like good work. An estimator is asked for a range and starts from the number already in the plan. The committed figure becomes the centre, a plausible margin is applied to each side, and the result is a tidy symmetric band around the commitment. Every downstream step then behaves impeccably: the sampling is sound, the percentiles are computed correctly, the curve is drawn to a professional standard. What comes out is the original optimism with error bars attached, and it is more dangerous than the point estimate was, because it now carries the borrowed authority of a probabilistic analysis.

The symptom is diagnostic and takes one glance to spot. Open the input table and compare each range with the planned value: if the plan sits almost exactly at the centre of range after range, the ranges were not elicited, they were decorated. A second tell is uniform width, where every input is given the same proportional band regardless of how much is genuinely known about it, which says the widths came from a convention rather than from evidence. A third is that no range is asymmetric, even though most real development inputs have far more room to go badly than well: a task can take three times as long as planned and almost never a third as long.

The fix has three parts and none of them is technical. Elicit the extremes first, asking for the plausible worst case and its cause before anybody says a central number, because starting at the ends stops the anchor forming. Calibrate the resulting range against what comparable efforts actually did rather than against what this one intends to do, which is the whole purpose of reference class forecasting. And treat a plan value sitting near the optimistic edge of the final range as the normal case rather than as an insult to the team, because across the recorded history of large projects that is where committed numbers usually turn out to have been. The literature on optimism bias explains why this feels so wrong to the people closest to the work.

Nine out of ten - megaprojects run over budget, which is the empirical reason the honest centre of a cost range usually sits above the number already written into the plan.

Source: Bent Flyvbjerg, The Iron Law of Megaproject Management

Quantifying only what is easy to quantify

Quantification follows data, and data is unevenly distributed across a plan. Unit costs, historical throughput, headcount, build durations for work the team has done several times before: these have clean numbers attached, so they get careful ranges and a lot of attention. The uncertainties that actually threaten the plan usually have no clean numbers at all. Whether the regulator approves on the first submission, whether a novel integration behaves at production volume, whether anybody wants the product at the price in the model, whether the one engineer who understands the legacy system stays another year: these resist distributions, so they quietly fall out of scope. The model then answers a narrower question than the one being asked, and answers it with conspicuous confidence.

The symptom is a result whose spread looks implausibly tight next to the team’s own intuition. When the analysis says the programme is very likely to land within a few percent of its budget and everyone in the room privately expects worse, the usual explanation is not that the room is pessimistic. It is that the inputs genuinely threatening the plan are not in the model. The quantified uncertainty is real as far as it goes, but it is the uncertainty of the well-understood part of the plan. The remaining exposure has not been removed, only made invisible, and invisible exposure does not show up in a contingency figure.

One in six - projects in a study of about 1,500 IT projects was a “black swan” with a cost overrun of around 200 percent and a schedule overrun of around 70 percent, and that exposure lives in the uncertainties nobody modelled.

Source: Harvard Business Review

The fix is not to force a distribution onto a political question. It is to name the unquantified uncertainties explicitly alongside the result, in the same document and on the same page, so the reader knows what the number covers and what it does not: this figure assumes first-pass regulatory approval and the current integration partner, and it prices neither. Stated that way, the omissions become agenda items rather than silent assumptions, and the conversation moves on to how to buy information about them. Some can then be pulled back into the model as discrete events with a judged probability and a stated consequence, which is cruder than a fitted distribution and considerably better than exclusion, as the worked business plan risk assessment illustrates.

Treating the P50 as a promise

A median is not a plan. The P50 is the value half the simulated outcomes fall below and half exceed, which makes it a useful internal working target and a poor external commitment: promising a P50 date is promising a coin flip. Organisations make this mistake because the P50 looks like the answer. It is the middle, it is the figure the model appears to be centred on, and it is the smallest number that can be defended in a room without sounding defeatist. So it becomes the budget and the board date, and the organisation then spends the next two years experiencing entirely ordinary variance as failure.

The downstream effects are cultural as much as financial. When half the plans in a portfolio are funded at their medians, roughly half of them overrun by construction, and overrun becomes the normal condition. Teams learn that dates are theatre, estimates acquire private padding that is invisible at portfolio level, and the honest estimator is punished relative to the colleague who quietly buffers. The organisation has effectively chosen a fifty percent service level without ever deciding to choose one, because nobody stated a confidence level at any point in the process.

The fix is to separate the internal working target from the external commitment and to fund the commitment at a stated confidence level. Public programmes have formalised exactly this. NASA requires that its larger projects plan and budget to a seventy percent joint confidence level, a single probability covering cost and schedule together, with any lower figure justified in the decision record. The federal cost estimating guide shows the same arithmetic from the other side, pricing a programme at the fifty percent confidence level and again at eighty percent. The discipline worth copying is not the particular percentage; it is naming one at all, since a plan funded at a confidence level nobody stated is funded at a confidence level nobody chose. Teams running go or no-go decisions should settle that threshold before they see the output rather than after.

Precision theatre

Arithmetic launders judgement. Inputs elicited in a twenty minute conversation go into a simulation, and what comes back out is a probability of 73.4 percent and a P80 of 41.7 million (illustrative), reported to a decimal place that no part of the input chain can support. The numbers are not wrong in the sense of being miscalculated, because the computation is exact. They are wrong in what they imply about how much is known. The decimal point borrows the authority of measurement for a set of estimates, and readers respond to the format in front of them rather than to the provenance behind it.

The symptom is that nobody asks where the inputs came from. A result expressed as roughly seven in ten invites the question naturally; a figure quoted to a decimal place closes it down, because the form of the number signals that the work has already been done to a standard beyond challenge. The same effect appears in comparisons. Two plan variants whose probabilities differ by a single point get ranked as though the difference were real, when the gap sits far inside the noise of the ranges that produced it, and a committee then spends an hour choosing between two options it cannot actually distinguish.

The fix is cheap. Round the output to the precision the inputs deserve, and say where the judgement came from. Roughly seven in ten (illustrative), with the delivery ranges elicited from the engineering leads and the market assumptions supplied by the commercial team, is both more honest and more useful to a decision maker than any decimal. The measurement tradition has carried this convention for decades, carefully distinguishing uncertainty evaluated statistically from uncertainty evaluated by judgement and insisting that the stated interval and its coverage travel with the result rather than being left to the reader to infer. Our longer treatment of the hidden costs of false precision sets out how much decision quality this one habit recovers.

One analysis, never revisited

Quantification is often procured as a gate artefact. A programme needs a probabilistic analysis in order to pass approval, the analysis is produced, the gate is cleared and the file is closed. Within weeks the plan has moved: the long-lead order is placed, the pilot reports, the second supplier drops out, the scope is trimmed to protect the date. The distribution in the approved document still describes a world that no longer exists, and the longer it stays on the slide the more authority it accumulates, because it remains the only quantified thing anybody in the programme has to point at.

The underlying point is that not all uncertainty is permanent. The standard treatment separates the part that is irreducible variability from the part that merely reflects what the modeller does not yet know, and the distinction turns on whether more information would reduce it. A development plan is dense with the reducible kind: the integration either works at volume or it does not, and a two week spike settles the question. Every time one of those resolves, the correct distribution is narrower than the one before it. A single analysis cannot represent that movement, and a plan that never re-runs is carrying a wider range than it has earned, which is its own quiet form of waste.

The fix is to attach re-running to events rather than to the calendar. When a material assumption resolves, re-run. When a range turns out to have been wrong, re-run and say so. Before each steering meeting, re-run. Expect the distribution to tighten over the life of the programme, and treat a distribution that is not tightening as a signal that the team is learning nothing from its own execution. All of this is only affordable if the second run is cheap, which is why re-run cost belongs in the procurement criteria rather than in the footnotes, and why uncertainty-first planning is a practice rather than a document.

A distribution nobody reads

A technically flawless output can still fail completely at the only point where it matters. A histogram with a fitted curve, an S-curve with percentile markers and a table of summary statistics is the correct representation of the analysis and the wrong artefact for an audience that has eleven minutes and needs a sentence. What happens next is entirely predictable: the chart is acknowledged politely, nobody interrogates it, and the meeting reverts to the point estimate on the previous slide, because that is the only number in the room anyone present can repeat afterwards.

The symptom is a decision that proceeds exactly as it would have done if the analysis had never been commissioned. Ask the attendees afterwards what the probability of clearing the budget was and nobody can say; ask what the top driver was and you get the risk everyone was already worried about rather than the one the model ranked first. This is usually diagnosed as an audience failure and it is almost always a presentation failure. The analyst has shown the method instead of the conclusion, and the method is of genuine interest to perhaps two people in the room.

Lead instead with the probability of clearing the threshold and the three drivers that most move it, expressed in the units the audience already uses, and keep the chart behind that as support for whoever wants to go deeper. One sentence is usually enough: we are about seven in ten to land inside the approved budget (illustrative), the exposure is concentrated in the integration schedule, the second supplier and the hiring ramp, and two of those three are things we can act on this quarter. Then show the curve. Our guidance on running risk conversations with non-technical leaders covers the sequencing in more detail, and the sequencing matters more than the quality of the chart.

Using the model to win an argument

The last failure is the most corrosive, and it rarely involves anybody lying. A decision has effectively been taken. An analysis is commissioned to support the approval, and the first result is unhelpful: the probability of clearing the budget is too low to present. So the inputs are revisited, in entirely defensible language. That worst case was unduly pessimistic. The contingency on the integration double-counts a risk already held centrally. The adoption range ought to reflect the new partnership. Each adjustment is arguable on its own merits, every one of them happens to move the answer in the same direction, and after four of them the model supports the decision.

This is self-defeating in a way worth stating plainly, because it is the whole reason to care about the practice. The only thing that made the number worth having was that it came out of a process indifferent to what anyone wanted it to say. Once the inputs have been tuned towards the conclusion, the output stops being evidence and becomes rhetoric with a chart attached, which is more damaging than no analysis at all: it closes down the challenge an honest point estimate would have invited, and it spends whatever credibility the organisation had built in quantification on a single decision. The tell is simple and almost always present: a disappointing first result followed by input revisions that all move one way.

Three practices prevent it and all of them are cheap. Record the inputs, with their ranges and the name of whoever supplied them, so that a later revision is visible as a revision instead of being absorbed silently into the next version. Have the ranges reviewed by somebody who did not write the plan, because the person carrying the commitment cannot also be the sole source of the pessimistic tail. And score forecasts against outcomes over time, so that a habit of flattering inputs eventually shows up as a pattern rather than being re-argued case by case; a plain record comparing the probabilities a person stated against the outcomes they later reported, which is what calibration tracking provides, is enough to make that pattern visible to the estimator themselves. None of this is a control problem to be solved with permissions. It is a norm, and the work of building one is covered in our guide to a risk-aware culture and in the practical comparison of risk forecasting software.

Uncertainty quantification applied: four kinds of development plan

The vocabulary of uncertainty quantification is deliberately domain neutral, which is a strength while you are learning it and a weakness the moment you have to apply it to a plan with a board date attached. The mechanics are identical across the four families of development plan that most organisations actually run, but the place the uncertainty hides is not, and neither is the question the quantified answer is meant to settle. What follows walks the same four steps through each family: the output the decision genuinely turns on, the inputs that carry the uncertainty, which of those inputs are aleatory and which are epistemic, and what changes about the decision once the answer arrives as a distribution rather than as a number.

Product development plans

A product development plan almost always resolves to one of two outputs. Either the decision turns on a launch date measured against a window that has already been committed to somebody - a trade show, a regulatory deadline, a customer contract, a competitor launch the commercial team is pricing against - or it turns on a contribution margin measured against an internal hurdle, in which case the date matters only insofar as it shifts the revenue curve. Those two outputs demand different analyses, and the first discipline of quantification is refusing to run anything until the team has said which one it is. A plan quantified against a vague aspiration to “launch in the first half and make money” produces a distribution nobody can act on, because there is no threshold to count scenarios against.

The inputs that carry the uncertainty in a product plan are reasonably stable across companies. Development effort is the obvious one, and the one teams are most practised at arguing about, but it is rarely the widest. Integration risk tends to be wider, because the number of unknown interactions between a new component and the systems it has to sit inside is a function of how well those systems are understood, and that is usually worse than anyone admits in a planning meeting. Dependency on a scarce team is wider still, since it couples otherwise independent workstreams through a single queue, and a queue converts small variations in demand into large variations in wait. Certification, legal review, security review and app store review each contribute a delay distribution with a long right tail and almost no left tail, because these processes can take far longer than expected but essentially never finish early.

Adoption sits apart from all of those, because it does not affect the date at all; it determines whether hitting the date was worth anything. Separating the inputs into aleatory and epistemic parts is what makes the analysis actionable rather than merely honest, and adoption is the clearest case. Adoption uncertainty is overwhelmingly epistemic: it is wide because the organisation does not yet know how the market will respond, and that ignorance is reducible. A pilot, a limited regional release, a pre-order window or a paid concierge version of the product all buy information, and the right way to think about their cost is as the price of narrowing a distribution before the irreversible spend begins rather than as a delay to the real work.

Development throughput variation is a different animal. Some of it is epistemic, in that a team which has never built this kind of component carries a wider estimate than one that has built five, and that gap closes with a spike or a prototype. But a large residue is simply the week to week variability of knowledge work, and no amount of further study removes it. That residue has to be carried as float in the schedule rather than argued away in a review, and the most common planning failure in product organisations is treating it as though it were epistemic: commissioning another estimation workshop in the hope that the range will narrow, when the honest response is to decide how much float the committed window deserves and to price it.

The practical consequence is that the two halves of the plan get different treatments, and the quantified view is what makes them comparable. Suppose a team quantifies a launch and finds that the committed window is cleared in 58 percent of simulated scenarios (illustrative), that pulling one integration workstream off the shared platform team lifts that figure to 74 percent (illustrative), and that a four week pilot ahead of full launch narrows the margin distribution enough to push expected contribution above the hurdle in most runs (illustrative). Those are three different decisions: one buys schedule confidence with headcount, one buys information with time, and one does nothing and accepts the odds as they stand. Generating those alternatives explicitly rather than debating them in the abstract is the point of comparing plan variants, and the simulation machinery behind the comparison is covered in our guide to Monte Carlo simulation for business.

One further point is specific to product plans and routinely missed. The date and the margin are not independent outputs, because most of the levers that protect the date degrade the margin, and most of the levers that protect the margin put the date at risk. Adding contractors, paying for expedited review, descoping to a thinner first release and buying adoption with introductory pricing all trade one output against the other. Quantifying them separately and then comparing two point answers hides the trade entirely. Quantifying them jointly, so that every simulated scenario carries both a date and a margin, shows the shape of the trade directly: how much margin a given level of schedule confidence costs, and whether there is any version of the plan in which both thresholds are cleared at the confidence the business says it wants.

Capital projects and construction programmes

Capital programmes are the most mature domain for uncertainty quantification, partly because the money is large and visible and partly because the failure record is so well documented that denial became untenable. Two outputs carry the decision. The first is total installed cost measured against an approved budget, which is the number the capital committee owns and the number that determines whether the project gets funded at all. The second is mechanical completion or commissioning measured against a date, which is the number the operating business owns, because the date is when revenue, capacity or regulatory compliance actually begins. The two are linked through time related costs, and a plan that quantifies only one of them has quantified half a problem.

The base rate here is unambiguous. McKinsey has reported that large construction projects typically run up to 80 percent over budget and take around 20 percent longer than scheduled, a pattern stable enough across decades and geographies that it functions as a prior rather than as a cautionary tale. The KPMG Global Construction Survey found that only 31 percent of respondents reported their projects coming within 10 percent of budget, which is the same finding stated from the other direction: roughly two thirds of owners cannot hold a capital plan inside a 10 percent band. Against numbers of that size, a single point estimate presented to an investment committee is not a forecast at all; it is the optimistic end of a distribution nobody has drawn.

80% - is the upper end of the budget overrun McKinsey reports for large construction projects, which also tend to take around 20% longer than scheduled.

Source: McKinsey & Company

31% - of respondents to the KPMG Global Construction Survey reported that their projects came within 10% of budget.

Source: KPMG Global Construction Survey

The more instructive figure sits in McKinsey’s work on megaprojects. Looking at rail, it found cost overruns averaging about 44.7 percent while demand was overestimated by 51.4 percent. Read those two numbers together and the conventional framing of capital risk inverts. The cost side was wrong by a large margin, which is what everybody expects and what every contingency process is built to absorb. The benefit side was wrong by a larger one, and almost nothing in a standard capital governance process is built to absorb that. A rail scheme delivered exactly on budget while carrying half its forecast ridership is a worse outcome for the sponsor than one that overran by a third and filled its trains, and no amount of cost contingency protects against the second failure mode.

51.4% - is the average amount by which demand was overestimated in the rail megaprojects McKinsey examined, against cost overruns that averaged about 44.7%.

Source: McKinsey & Company

That asymmetry describes the single most common gap in capital uncertainty analysis as it is actually practised. Contingency setting is a mature discipline: estimators build three point ranges on quantities and rates, propagate them, and report a figure at a stated confidence level. The benefit side of the same business case is typically a deterministic row in a spreadsheet, carrying a throughput, a tariff, a utilisation rate or an occupancy assumption with no range on it whatsoever. The result is a plan whose cost is quantified to a confidence level and whose value is merely asserted, and the asserted half is empirically the more wrong half. Putting a distribution on demand, price and utilisation is less comfortable than putting one on steel tonnage, because the evidence is thinner and the assumptions are more political, but discomfort is not a reason to leave the larger error unmeasured. The governance habits that let this persist are worked through in more detail in our guide to capital project risk.

The aleatory and epistemic split is unusually clean in capital work, which is part of why the domain is such a good teacher. Weather downtime, labour productivity variation, equipment availability and the dispersion of unit rates within a known market are aleatory: they vary run to run, historical records characterise them well, and no further study collapses them. Ground conditions before the boreholes are drilled, the completeness of the scope definition at the point of sanction, the permitting pathway and the demand forecast are epistemic: each can be narrowed deliberately, by survey, by front end engineering, by early regulator engagement, by market study. Those are purchasable reductions in uncertainty, and deciding which of them to buy before sanction is precisely the decision a quantified view supports and a point estimate cannot even frame.

Software and technology programmes

Software programmes nominally resolve to the same two outputs as capital projects, a cost and a date, but in practice the decision usually turns on a third thing that is far harder to pin down: whether the capability arrives in a state that delivers the value the business case promised. A platform migration that lands on time and on budget and still leaves the organisation unable to decommission the system it was supposed to replace has failed, and that failure is invisible to any measure watching only spend and schedule. So the output set for a technology programme is cost, date and realised value, and the third is the one that most often decides whether the programme was worth running at all.

The record is worse than most sponsors believe. McKinsey, working with the University of Oxford, studied 5,400 large IT projects and found that 45 percent ran over budget, 7 percent ran over time, and the portfolio delivered 56 percent less value than predicted, with 17 percent going so badly that they threatened the existence of the company. The ordering of those numbers is the finding. Cost overruns were common, schedule overruns were comparatively modest, and the value shortfall dwarfed both. Separately, Flyvbjerg and Budzier reported in Harvard Business Review on roughly 1,500 IT projects, finding an average cost overrun of 27 percent and, more significantly, that one in six was a black swan with a cost overrun of around 200 percent and a schedule overrun of about 70 percent. An average of 27 percent sitting in front of a tail like that is not a planning assumption; it is a warning that the mean is the least useful summary of the distribution.

56% - less value than predicted was delivered across the 5,400 large IT projects McKinsey studied with the University of Oxford, where 45% also ran over budget.

Source: McKinsey & Oxford

1 in 6 - IT projects in a study of roughly 1,500 was a black swan, with a cost overrun of around 200% and a schedule overrun of about 70%.

Source: Harvard Business Review

Software is structurally harder to quantify than concrete, and pretending otherwise produces false precision rather than better decisions. Two reasons dominate. The first is that the work is novel by definition: if the component already existed in the form required, the sensible decision would be to buy or reuse it, so the tasks that remain are exactly the ones with the least relevant history behind them. Reference classes do exist, and they are worth building, but they are coarser and the matching is far more arguable than it is for the twentieth bridge of a given span. The second reason is that scope in software is negotiable in a way that scope in construction is not. A bridge that is two thirds built is not a bridge. A platform that is two thirds built is often shippable, and that optionality is genuinely valuable, but it means the thing being estimated moves while the estimate is being made, and a distribution over a moving target is a weaker instrument than a distribution over a fixed one.

The response is not to abandon quantification but to change what gets quantified. Schedule and value should be modelled separately, with the uncertainty on the value estimate kept explicit rather than folded into a single risk adjusted number, because the two are driven by different things and reduced by different actions. The 56 percent value shortfall is the strongest available argument for putting a range on the benefit side of a technology business case rather than only on the cost side: if the historical error in value is larger than the historical error in cost, a business case with a cost range and a single point benefit has been quantified on the wrong axis. In practice that means naming the two or three value drivers the benefit actually depends on, giving each a defensible range, and reporting the probability that the programme clears its value threshold alongside the probability that it clears its budget.

The aleatory and epistemic distinction still earns its keep here, though the balance shifts heavily toward the second pile. Throughput variation across iterations, defect discovery rates and the dispersion of effort across similar tasks behave aleatorically and are measurable from a team’s own history. Almost everything else is epistemic: whether the legacy data will migrate cleanly, whether the third party interface behaves as documented, whether the regulator accepts the control design, whether users adopt the new workflow. Each of those can be narrowed before the main commitment with a spike, a data profiling exercise, a sandbox integration or a pilot cohort, and the sequencing question of which unknown to retire first is answered by whichever one the outcome turns out to be most exposed to. The mechanics of running this on a delivery plan are covered in Monte Carlo simulation for project management, and the behavioural reasons these programmes keep failing in the same ways are set out in why projects fail and how to beat the odds.

Commercial and market-entry plans

Commercial and market entry plans resolve to financial outputs rather than physical ones, and the three that matter are payback period, peak cash requirement and net present value measured against a hurdle rate. Peak cash is the one most often omitted and the one most likely to end the venture, because a plan can carry an attractive net present value and still fail in month fourteen if the trough of the cash curve is deeper than the funding available to cover it. Quantifying the plan therefore means quantifying the shape of the cash curve rather than just its endpoint, and the useful question is not what the payback period is but how deep the hole gets across the range of plausible demand paths, and how confident anyone is that it can be funded.

Almost all of the uncertainty in a commercial plan is epistemic, and almost all of it is about demand. Cost structures in a new market are knowable with effort: rents, wages, logistics, duty rates and compliance costs can all be researched down to a tight range before anyone commits capital. Demand cannot be researched to anything like the same tightness, because it depends on how real customers behave in response to an offer that does not yet exist. That asymmetry is the whole structure of the problem, and it goes a long way toward explaining why the base rate for commercial plans is as poor as it is. The US Bureau of Labor Statistics finds that about 20 percent of new business establishments fail within their first year and about half are gone by year five, a survival pattern that has been remarkably stable across decades and locations. Those are not plans that got their cost assumptions wrong by a few points.

About half - of new business establishments have failed by year five, with roughly 20% failing within their first year, in US Bureau of Labor Statistics data.

Source: US Bureau of Labor Statistics

The second structural feature of commercial plans is that their drivers are correlated, which matters enormously for how they are modelled. PwC’s Global CEO Survey finds macroeconomic volatility to be the leading external threat chief executives cite, and the reason that single answer dominates is that macroeconomic conditions move several plan inputs at once. A downturn does not politely reduce one line in the model: it suppresses demand, lengthens sales cycles, stretches receivables, raises the cost of capital and softens pricing power simultaneously. Sampling each of those independently produces a distribution that is far too narrow in the tails, because the simulation never generates the scenario in which everything goes wrong together, which is precisely the scenario that kills commercial plans. Any honest quantification of a market entry has to let the demand side variables move together, and the practical test of whether a model does so is whether its worst scenarios look like the bad quarters the business has actually lived through.

What the quantified answer changes about a commercial decision is mostly the funding conversation. A point estimate invites a binary approval: the plan either clears the hurdle or it does not, and the discussion becomes an argument about whether the central case is credible. A distribution reframes the question as how much capital to commit against how much confidence, and it makes staged commitment the obvious structure rather than a sign of weak conviction. If the probability of clearing the hurdle is moderate but the peak cash requirement at a high confidence level is affordable, the plan is fundable with a tranche structure and a defined kill criterion. If the peak cash requirement at that confidence level exceeds what the business can carry, the plan is not fundable at the scale proposed, however attractive its central case looks, and that is a conclusion a point estimate structurally cannot reach.

There is one more discipline that commercial plans need more than the other three families, which is explicit treatment of the thing being sold before the market has voted on it. Price, attachment rate, churn and sales cycle length are all guesses dressed as parameters at the point of approval, and each of them can be bought down cheaply relative to the capital at stake: a pricing test, a letter of intent, a paid pilot, a small geography opened early. Worked examples of putting ranges on exactly these inputs are set out in our business plan risk assessment example, and the broader practice of testing a commercial plan against alternative demand paths rather than a single forecast is covered under what if analysis.

Making quantified uncertainty stick inside an organisation

The analytical content of uncertainty quantification is the easy part. Teams pick up three point estimation, learn to read an S curve and learn to interpret a ranked list of drivers inside an afternoon. What fails is adoption: the first quantified plan lands well, the second is treated as an exception because the deadline was tight, and within two quarters the organisation has drifted back to single numbers with a risk register bolted on the side. The practices that prevent that drift are organisational rather than mathematical, and they are considerably smaller than most change programmes assume. There are five of them, and none requires a methodology document.

Someone has to own the range

The ranges on a plan have to belong to the person who owns the plan. This sounds like an organisational detail and is in fact the hinge on which everything else turns. A range produced by a central analytics group, however well constructed, is a number the plan owner never agreed to, and the moment it becomes inconvenient - at the review where the optimistic end is what the room wants to hear - it will be disowned as somebody else’s model. A range the plan owner built, defended and signed is a commitment. The analytics function should supply the method, the tooling, the reference classes and the challenge, and should resist every invitation to supply the numbers themselves.

The governance consequence is specific and worth stating plainly: the reviewer’s job is to challenge the width of the range, not to replace it. A reviewer who substitutes their own figure has converted an estimate into a negotiation and taught the organisation that ranges are opening bids. A reviewer who asks why the range is that wide, what evidence sets each of its edges, and what action would narrow it is doing the thing that actually improves estimates, because it forces the owner to articulate the reasoning behind the distribution rather than the value at its centre. Over a few review cycles this changes what people bring to the meeting: they stop arriving with a defended number and start arriving with a defended range, which is a far more honest artefact and a far more useful one.

One question does more work at a review than any other: what would have to be true for the optimistic end to happen? It is a better question than its pessimistic twin, because teams are well practised at narrating disasters and much less practised at listing the specific conditions their own best case silently assumes. Asked properly, it surfaces the hidden conjunction - that the optimistic schedule requires the vendor to deliver early, the hiring plan to complete on time, the regulator to approve first pass and no competing priority to take the shared team, all at once - and the moment that list is spoken aloud, everyone in the room understands why the optimistic end is improbable rather than merely ambitious. It also tends to generate the action list for free, because several of those conditions turn out to be purchasable.

Write the decision down

The cheapest high value habit in this entire discipline is writing the decision down at the moment it is taken. A decision record does not need to be a document or a process. It needs five things: the threshold the plan was judged against, the probability of clearing it at the time of the decision, the top drivers the analysis identified, the confidence level the organisation chose to fund, and the actions that were made conditions of approval. That is a paragraph. It can live in the approval minute, in the ticket or in the plan itself, and it takes less time to write than the meeting took to schedule.

What the paragraph buys is institutional memory, and specifically the ability to tell a bad decision from a bad outcome. Without it, a programme that overruns gets reconstructed after the fact by people who remember the point estimate and nothing else, and the conclusion is always that the estimate was wrong. With it, the organisation can see that the plan was approved at a 55 percent probability of clearing the date (illustrative), that the two drivers which moved are exactly the two the analysis flagged, and that two of the four conditions of approval were quietly dropped in the first quarter. That is a different post mortem with different lessons, and it is the only version that produces learning rather than blame.

The record also disciplines the decision at the time it is taken, which is arguably the larger benefit of the two. It is uncomfortable to write down that a programme was funded at a modest probability of success, and that discomfort is informative: it prompts either a better plan, a wider range, a staged commitment, or an explicit and owned acceptance of the risk. Organisations that cannot bring themselves to write the sentence are usually organisations that would not have liked what the sentence said. Over time, a shelf of these paragraphs becomes the most valuable estimating asset the business has, because it is the only record anywhere that pairs what was believed at the point of commitment with what actually happened afterwards.

Calibration is a personal discipline before it is an institutional one

Quantified uncertainty only has value if the probabilities mean something, and the only way to find out whether they do is to score them against outcomes. The established approach is Brier scoring, which has one property that makes it the right instrument for planning: it rewards being both confident and right, and it punishes confident error considerably more harshly than it punishes hedged error. That asymmetry is the whole point. A forecaster who says ninety percent and turns out to be wrong takes a much larger penalty than one who said sixty percent and was wrong, so the scoring rule removes the incentive to project certainty for rhetorical effect, while still refusing to reward the person who answers every question with a shrug at even odds. It punishes overconfidence and useless hedging at the same time, which is exactly what a planning culture needs.

The critical property, and the one most often misunderstood, is that calibration is a property of a sequence of forecasts and never of a single one. A single eighty percent forecast that fails tells you nothing at all: it was supposed to fail one time in five. Only across dozens of forecasts does a pattern emerge, and the patterns that emerge are personal and surprisingly consistent. Some people are systematically overconfident on schedules and well calibrated on costs. Some are well calibrated on their own team’s work and badly overconfident about anything depending on a third party. Some compress everything toward the middle and never commit to a strong probability even when the evidence clearly supports one. None of those patterns is visible from a single estimate, and all of them are correctable once the person can see their own.

That is why the improvement is personal rather than institutional. A score is not a performance metric to be rolled up and reported; it is a mirror, and the value comes from an individual noticing the shape of their own error and adjusting the next estimate accordingly. Incertive supports the habit with calibration tracking: a record comparing a user’s own probability estimates against the outcomes they report, so that over a series of forecasts they can see where their own judgement runs hot or cold. The noticing is still theirs to do, and the adjustment happens in the estimator’s head rather than anywhere in the software, which is as it should be, because the habit is the thing that matters and the record exists only to make the habit possible.

The language that makes a probability usable

A quantified plan is only as useful as the language available to discuss it, and most organisations need a shared vocabulary of four items rather than forty. The threshold: the specific thing the plan has to clear. The probability: the share of simulated scenarios that clear it. The confidence level funded: the point on the distribution the organisation chose to resource. The top three drivers: the variables the outcome is most exposed to. A team that can state those four things about any plan has everything it needs to take a decision, and a team that insists on more has usually confused thoroughness with communication.

The two failure modes sit at opposite ends and are both familiar. The technical register produces a document full of distribution names, correlation structures and confidence intervals that is comprehensively correct and comprehensively unread, and its practical effect is to push the decision back to whoever can summarise it in a sentence, usually without the caveats. The vague register produces a colour coded grid and the phrase “medium risk”, which commits to nothing, cannot be shown to be wrong, and gives a reader no basis whatsoever for choosing between two plans. Both registers fail for the same underlying reason: neither states a threshold and a probability of clearing it, which is the one formulation that is both precise enough to act on and short enough to be read.

The sentences that work in a board pack are unglamorous and almost formulaic, which is their virtue. “We clear the committed date in 62 percent of scenarios; funding to 80 percent confidence requires either eight more weeks of float or the removal of the shared platform dependency” (illustrative). “Total installed cost is 240 million at the 50 percent confidence level and 268 million at 80 percent, and we are requesting approval at 80 percent” (illustrative). “The three variables that move this outcome most are commissioning duration, the tariff assumption and the permitting timeline, and we propose to retire the permitting uncertainty before sanction” (illustrative). Each of those names a threshold, a probability and an action, and each can be challenged on its substance rather than on its tone. That is what usable looks like, and it is learnable in a single review cycle.

It is worth noting how much the surrounding context has shifted, because the usual objection to this kind of language - that it is too specialist for a general management audience - is now dated. Gartner reported in 2025 that 59 percent of finance leaders use artificial intelligence in the finance function, and Deloitte’s Q4 2025 CFO Signals survey found that 87 percent of chief financial officers regard the technology as very or extremely important. Whatever one makes of the pace of that adoption, it establishes that the appetite for quantitative tooling inside planning and finance functions is now mainstream rather than the preserve of a specialist risk team, and that an audience long assumed to want a single number is increasingly comfortable being handed a distribution instead. The broader framing of that shift, from reporting numbers to structuring decisions, is set out in our introduction to decision intelligence.

59% - of finance leaders use AI in the finance function, in Gartner research published in 2025.

Source: Gartner

87% - of chief financial officers say AI is very or extremely important, in the Deloitte Q4 2025 CFO Signals survey.

Source: Deloitte

Start with one decision, not a programme

The last practice is the one that determines whether any of the other four ever happen. Do not roll uncertainty quantification out as a methodology. Pick the next decision that is large enough to matter and near enough to be real - a sanction paper due this quarter, a launch commitment, a market entry going to the board - and quantify that one. A single live decision has a sponsor who cares about the answer, a deadline that forces the work to completion, and a verdict that arrives soon enough for everyone involved to see whether the analysis helped. A methodology rollout has none of those three things, which is why rollouts produce templates and pilots produce converts.

The comparison that changes minds is between the quantified view and the memo the team would otherwise have written. The memo contains a number, a set of assumptions in an appendix and a paragraph of hedging. The quantified view contains a probability of clearing the stated threshold, the cost of moving that probability up, a ranked list of what the outcome is genuinely exposed to, and a short list of actions that narrow the uncertainty before the money is committed. Presented side by side to the same committee, the second document changes the character of the meeting, and that single experience does more to establish the practice than any volume of internal training. It also exposes, quickly and cheaply, whether the organisation is willing to act on a probability or merely to admire one.

There is a budget argument alongside the behavioural one. The same Deloitte Q4 2025 CFO Signals survey found that 43 percent of chief financial officers name cloud planning the top cost technology, which suggests the tooling budget for better planning is already being spent in most large organisations. What is usually missing is not software but the habit of stating a threshold and a probability against it, and a habit is learned on a live decision rather than installed by a programme. If you want to see what the finished artefact looks like before committing to anything, the walkthrough in how it works covers the sequence end to end, and the longer organisational arc, from one quantified decision to a culture that expects them, is set out in our piece on building a risk aware organisation.

43% - of chief financial officers name cloud planning the top cost technology, in the Deloitte Q4 2025 CFO Signals survey.

Source: Deloitte

Decide with the odds in front of you

The case for uncertainty quantification is not that it makes forecasts more accurate. It does not, and no method can: the future of a development plan is genuinely uncertain, and a distribution is an honest description of that fact rather than a way around it. The case is that quantifying uncertainty changes what you can see and therefore what you can decide. A point estimate hides the asymmetry of its own risk, the identity of the assumption that will break it, and the size of the buffer it would take to make the commitment safe. A distribution surfaces all three, and it does so before the money is committed rather than afterwards in a post-mortem.

The record on plans that skip the exercise is unusually consistent across industries, decades and project types, and the consistency is the clue. Overruns that run in one direction, at similar magnitudes, across hundreds of projects and thousands of programmes are not the signature of weak estimators. They are the signature of a method that cannot represent what it is being asked to represent. Replacing that method does not require a modelling team or a quarter of process design. It requires describing the handful of inputs that matter as ranges instead of points, propagating them, and reading the answer as a probability against a threshold you have named in advance.

Three habits carry most of the benefit. Sort the plan’s unknowns into the variation you have to carry and the ignorance you can pay to reduce, because that single distinction tells you where to add float and where to buy a prototype instead. Decide, explicitly and in writing, what confidence level you intend to fund, in the way that serious programme offices already do. And write the forecast down at the moment of the decision, with its threshold and its date, so that later you can find out whether your probabilities meant anything. None of the three needs permission from anyone.

What changes in practice is the conversation at the gate. A review built on a single number becomes an argument about whose optimism to believe, settled by seniority. A review built on a quantified plan becomes a decision about a measured quantity: here is the probability of clearing the line, here are the three drivers responsible for most of the risk, here is what it would cost to move the odds, and here is the confidence level we are prepared to fund. That is a better conversation not because it is more cautious but because it is more decisive. Teams that can price their own uncertainty commit faster to the plans worth backing and stop the ones that are not while stopping is still cheap.

If you want to go deeper on the engine that does the propagating, the complete guide to Monte Carlo simulation covers it end to end, and the companion guides on Monte Carlo simulation software, Monte Carlo simulation for project management and three-point estimation go into the tooling, the project application and the elicitation technique respectively. To see the shape of the output on a real decision, run a quick pass with the Monte Carlo calculator or the project success calculator, read how the approach works on the methodology page, and when you want your own development plan quantified rather than asserted, get started. The next commitment is coming either way. Make it with the odds in front of you.

Frequently Asked Questions

What is uncertainty quantification?

Uncertainty quantification is the discipline of establishing how wrong an estimate could be, and with what likelihood, for a quantity you cannot know exactly. Applied to a development plan it replaces a single figure, such as a cost or a delivery date, with a measured range and a probability of clearing the threshold that matters. The workflow is consistent across fields: identify what is uncertain, characterise each unknown as a range rather than a point, propagate those ranges through the plan, and report the result as a probability a decision-maker can act on.

What is the difference between aleatory and epistemic uncertainty?

Aleatory uncertainty is irreducible variation: day-to-day swings in build throughput, weather, which particular customers convert this month. No amount of study removes it, so it has to be carried with float, buffer or spare capacity. Epistemic uncertainty is ignorance that can be reduced: whether an integration will work, how a regulator will rule, what a supplier will actually charge. It shrinks when you buy information through a prototype, a pilot, a survey or a firm quotation. The distinction is the standard one in the reliability literature, and it matters because a plan that pads where it should investigate wastes money, while a plan that investigates where it should pad misses dates.

How is uncertainty quantification different from a risk register?

A risk register lists named risks with a likelihood and impact rating, usually on a coarse scale, and it is useful for assigning ownership and tracking mitigation. What it cannot do is arithmetic: there is no mechanism connecting the entries on the list to the outcome, so a register cannot tell you the probability that the plan comes in under budget. Uncertainty quantification works the other way round. It describes the plan’s inputs as ranges, runs them through the plan’s own logic, and produces a distribution of outcomes from which any probability question can be read. The two are complements, not substitutes.

What methods are used to quantify uncertainty?

The workhorse for characterising an input is three-point estimation, which asks for a minimum, a most likely value and a maximum, because that is the format experts can actually answer. Those ranges are then propagated through the plan with Monte Carlo simulation, which places no restriction on the structure of the model. Sensitivity analysis, usually presented as a tornado diagram, ranks which inputs move the outcome most. Reference-class forecasting calibrates the ranges against what comparable efforts actually did. The US Government Accountability Office’s cost estimating guide describes uncertainty analysis as being conducted through a three-point estimate methodology, a risk-driver approach, or a combination of both.

What confidence level should a development plan be funded at?

There is no universal answer, but there is a useful precedent: NASA policy for projects and single-project programmes with a life-cycle cost of $1 billion or more directs mission directorates to plan and budget based on a 70 percent joint cost and schedule confidence level, or as approved by the decision authority, and requires a written justification whenever a programme is budgeted below that level. The transferable practice is not the specific figure but the explicitness. Decide what confidence level you intend to fund, state it, and document the reasoning if you choose to carry more risk than that.

Put a probability on your next development plan

Describe the plan in plain language and Incertive identifies the uncertainties, runs the simulation, and returns a probability of success, the ranked drivers behind it, and a clear go, no-go or conditional call.

Quantify My PlanBack to Blog