Business plans fail at alarming rates, and the reasons are well-documented in decades of academic research. This article examines the evidence - from Kahneman and Tversky's planning fallacy to Flyvbjerg's megaproject database to the Standish Group's IT project data - and explains what successful projects do differently.
Business plans fail. This is not a controversial statement; it is one of the most thoroughly documented empirical regularities in organizational research. Whether measured by cost overruns, schedule delays, missed revenue targets, or outright project cancellation, the majority of business plans do not achieve what they set out to achieve. This pattern holds across industries, geographies, organization sizes, and time periods. It holds for IT projects and construction projects, for startups and Fortune 500 companies, for public sector programs and private sector initiatives.
The consistency of this pattern is itself informative. If plan failures were random - caused by bad luck, unusual circumstances, or one-off mistakes - we would expect the failure rate to be symmetric: roughly as many projects would come in under budget as over budget, and as many would finish early as late. Instead, the data shows a dramatic skew toward overruns and delays. Projects almost always cost more and take longer than planned, and they almost never cost less or finish earlier. This asymmetry is the signature of a systematic bias rather than random error.
Understanding why plans fail is not merely an academic exercise. Every failed plan represents wasted resources, missed opportunities, damaged credibility, and organizational disappointment. More importantly, the same cognitive and organizational factors that cause plans to fail can be identified, measured, and counteracted - if the organization is willing to confront the uncomfortable reality that its planning processes are systematically biased. The planning fallacy is the most fundamental of these biases, and understanding it is the first step toward more realistic planning.
The economic consequences of plan failure are staggering. The Project Management Institute's 2018 Pulse of the Profession report estimated that organizations waste $1 million every 20 seconds, or roughly $2 trillion per year globally, due to poor project performance. While this figure should be treated as an order-of-magnitude estimate rather than a precise calculation, it reflects the enormous scale of the problem.
Beyond the direct financial costs, plan failure imposes significant indirect costs. Failed projects consume management attention that could be directed to more productive activities. They damage organizational credibility, making it harder to secure funding and support for future initiatives. They demoralize teams, leading to talent attrition and reduced engagement. And they create opportunity costs: every dollar and every hour invested in a failing project is a dollar and an hour that cannot be invested in a succeeding one.
Perhaps most insidiously, plan failure creates a cycle of distrust. When plans consistently fail to meet their targets, stakeholders learn to discount future plans. Budget requests are inflated because everyone knows they will be cut. Timelines are padded because everyone knows they will slip. Commitments are doubted because everyone knows they will not be met. This cycle of distrust makes it harder for genuinely realistic plans to be taken seriously, because they are evaluated in a context where all plans are assumed to be optimistic.
In 1979, Daniel Kahneman and Amos Tversky introduced the concept of the "planning fallacy" - the systematic tendency to underestimate the time, cost, and risk of planned actions while overestimating their benefits. The planning fallacy is arguably the single most important concept for understanding why business plans fail, and its implications for planning practice are profound.
Kahneman and Tversky distinguished between two perspectives on a plan: the "inside view" and the "outside view." The inside view focuses on the specific plan at hand: its unique characteristics, the steps required, the resources available, the team's capabilities. When planners take the inside view, they construct a narrative of how the project will unfold, step by step, from beginning to completion. This narrative naturally focuses on the plan for success - the sequence of events that needs to occur for the project to be completed on time and on budget.
The outside view, by contrast, focuses on the outcomes of similar past projects: the reference class. Rather than asking "How long will this specific project take?" the outside view asks "How long have similar projects typically taken?" The outside view does not require detailed knowledge of the specific project; it requires only knowledge of the statistical distribution of outcomes for comparable projects.
Kahneman illustrated the planning fallacy with a personal anecdote. In the 1970s, he participated in a curriculum development project with a team of experienced educators. When asked to estimate how long the project would take, the team members provided estimates ranging from 18 to 30 months. Kahneman then asked a colleague, Seymour Fox, who had extensive experience with similar curriculum projects, how long such projects typically took. Fox reported that similar projects had taken 7 to 10 years, and that 40% had never been completed at all. Despite hearing this outside-view data, the team proceeded with the inside-view estimate. The project eventually took 8 years to complete. As Kahneman later wrote in Thinking, Fast and Slow (2011), "We should have quit. None of us was willing to invest six more years of work in a project with a 40% chance of failure."
The inside view produces systematically optimistic estimates for several reasons that have been extensively documented in the cognitive psychology literature.
Focus on the plan rather than the base rate. When constructing a bottom-up estimate, planners focus on the specific steps required and estimate how long each step will take under normal conditions. This approach naturally excludes the myriad ways things can go wrong: unexpected technical difficulties, resource conflicts, scope changes, external dependencies, illness, turnover, supply chain disruptions, regulatory changes, and countless other factors that affect real projects. Each individual disruption may be unlikely, but the probability that at least one disruption will occur across a project with hundreds of activities approaches certainty.
Best-case scenario thinking. When estimating the duration of a task, people tend to imagine how long it would take under favorable conditions rather than average conditions. "If everything goes smoothly, this will take three weeks" becomes the estimate, even though everything rarely goes smoothly. The favorable-condition estimate is the lower bound, not the expected value, but it is routinely treated as the expected value.
Neglect of unknown unknowns. The inside view accounts for known risks but cannot account for risks that have not been identified. Donald Rumsfeld's distinction between "known unknowns" (risks we know about but cannot quantify precisely) and "unknown unknowns" (risks we have not even imagined) is directly relevant. Every project encounters surprises - events that were not anticipated in the plan. The inside view, by definition, cannot incorporate these surprises because they have not been imagined.
Motivational bias. Planners are often also advocates for the project. They want the project to be approved and to succeed. This motivation creates a subtle but powerful bias toward optimistic estimates: an optimistic plan is more likely to be funded, more likely to generate enthusiasm, and more flattering to the planner's competence. Even when planners are aware of this bias and try to correct for it, the correction is typically insufficient.
Kahneman's prescription for the planning fallacy is deceptively simple: use the outside view. Before relying on a bottom-up estimate, find out how similar projects have actually performed. If historical data shows that 70% of similar IT projects experienced cost overruns of 20% or more, a new IT project should incorporate that base rate into its estimates rather than assuming it will be different.
The outside view does not replace the inside view; it provides a reality check on it. If the inside view estimates a project cost of $5 million and the outside view (based on historical data from similar projects) suggests a median cost of $7 million, the discrepancy should be investigated. Perhaps the specific project has genuine advantages that justify the lower estimate. Or perhaps the inside view is subject to the usual optimistic biases. The comparison forces the planner to articulate why this project is different from the historical base rate - and if the reasons are not convincing, to adjust the estimate accordingly.
This approach is formalized in reference class forecasting, which Bent Flyvbjerg developed and advocated as a practical method for implementing Kahneman's outside view. Reference class forecasting is now recommended by several government agencies, including the UK Treasury and the Danish government, as a corrective for optimism bias in major project estimates. For more on this topic, see our detailed guide on the planning fallacy and how to counteract it.
Bent Flyvbjerg, a professor at the University of Oxford (and previously at Aalborg University in Denmark), has compiled the largest database of large project performance in the world. His research, spanning decades and published in numerous peer-reviewed papers and books, provides the most comprehensive empirical evidence for the systematic nature of planning failure.
Flyvbjerg, Holm, and Buhl (2002), in a paper published in the Journal of the American Planning Association, analyzed 258 large transportation infrastructure projects spanning 20 countries and five continents, with a combined value of approximately $90 billion. Their findings were striking:
The finding that overrun patterns have not improved over time is particularly damning. It means that decades of advances in project management methodology, scheduling software, cost estimation techniques, and organizational learning have not reduced the systematic bias in project estimates. The bias is not a technical problem that better tools can solve; it is a cognitive and organizational problem that requires a fundamentally different approach to planning.
Flyvbjerg identified two distinct causes of cost overruns: optimism bias (the unconscious tendency to underestimate costs) and strategic misrepresentation (the deliberate understatement of costs to get projects approved). His research suggests that both causes are at work, and that strategic misrepresentation may be the more important factor for large public projects.
Strategic misrepresentation occurs when project promoters deliberately understate costs and overstate benefits to make a project appear more attractive in the competition for funding. The incentive structure is clear: a project with a cost estimate of $500 million is more likely to be approved than the same project with a realistic estimate of $700 million. The promoter faces no personal consequence for the cost overrun (which will be discovered years later, often after the promoter has moved on), but faces an immediate consequence for a high estimate (the project may not be funded).
Flyvbjerg has described this dynamic as "survival of the unfittest": the projects that get funded are not the ones with the most realistic estimates but the ones with the most optimistic estimates. This creates a systematic selection bias in the project portfolio, where the projects that proceed are precisely the ones most likely to experience overruns. He has advocated for accountability mechanisms, independent estimation, and reference class forecasting as countermeasures.
In his 2023 book How Big Things Get Done (co-authored with Dan Gardner), Flyvbjerg expanded his database to include over 16,000 projects across diverse domains: IT, construction, energy, defense, space, Olympics, and more. The expanded analysis confirmed and extended his earlier findings:
Flyvbjerg's expanded research also identified characteristics of projects that perform well. Successful projects tend to: use modular designs (breaking the project into repeatable units), draw on extensive experience with similar projects, have strong governance and accountability, use reference class forecasting, and plan carefully before committing to execution ("think slow, act fast"). These characteristics are consistent with the outside-view approach that Kahneman advocated and with the probabilistic planning methods that Monte Carlo simulation enables.
The Standish Group, a research advisory firm, has published its CHAOS reports on IT project outcomes since 1994. The CHAOS reports are based on surveys and case studies of thousands of IT projects across industries and organization sizes. While the methodology has evolved over the years and some researchers have critiqued specific aspects of the Standish Group's approach, the reports provide the most widely cited dataset on IT project performance.
The original 1994 CHAOS report was a watershed moment for the IT industry. It found that only 16% of IT projects were completed on time, on budget, and with all planned features (categorized as "successful"). 53% were "challenged" (completed but over budget, over time, or with fewer features than planned), and 31% were "failed" (cancelled before completion or delivered and never used). These findings shocked the industry and catalyzed significant investment in project management practices, methodologies, and tools.
Subsequent CHAOS reports have shown improvement, but the overall picture remains sobering. The 2020 CHAOS report found that approximately 31% of projects were successful, 50% were challenged, and 19% were failed. The improvement in the "successful" category from 16% to 31% over 25 years is real but modest, and it may partly reflect changes in the definition of "success" (the Standish Group modified its success criteria in 2015 to include a broader measure of value delivery rather than strict adherence to the original plan).
One of the most important findings from the CHAOS reports is the relationship between project size and failure rate. Small projects (under $1 million) have dramatically higher success rates than large projects (over $10 million). The 2015 CHAOS report found that small agile projects had a success rate of approximately 58%, while large waterfall projects had a success rate of only about 6%. This relationship holds across methodologies: agile projects outperform waterfall at every size, but success rates decline with project size for both approaches.
This finding has a clear implication for planning: large, monolithic projects should be broken into smaller, more manageable components whenever possible. Smaller projects have shorter planning horizons, fewer dependencies, less accumulated uncertainty, and faster feedback loops - all of which contribute to more realistic planning and more successful execution. This is consistent with Flyvbjerg's finding that modular projects outperform monolithic ones.
The CHAOS reports have identified several factors that are statistically associated with project success. These include: executive support (projects with active executive sponsorship are more likely to succeed), clear business objectives (projects with well-defined, measurable objectives outperform those with vague goals), emotional maturity (the Standish Group's term for the interpersonal skills and trust within the project team), agile methodology (iterative approaches with frequent feedback outperform rigid, phase-gate approaches for software development), and modest scope (projects that start small and expand based on feedback outperform those that attempt a comprehensive solution from the start).
Notably, the CHAOS reports have consistently found that the planning methodology is one of the most important success factors. Projects that use formal estimation methods, risk analysis, and contingency planning outperform those that rely on informal estimates. This supports the case for decision intelligence approaches that bring quantitative rigor to the planning process.
The Project Management Institute (PMI), the world's largest professional association for project managers, publishes an annual survey called "Pulse of the Profession" that tracks project performance metrics across industries and geographies. The survey samples thousands of project management professionals and provides a broad-based view of project performance trends.
PMI's surveys have consistently found that a significant minority of projects fail to meet their original goals. The 2021 Pulse of the Profession report found that 11.4% of investment was wasted due to poor project performance. Organizations that were rated as "underperformers" in project management maturity completed only 46% of projects on time, 40% within budget, and 43% meeting original goals and business intent.
The PMI data also reveals significant variation in project performance based on organizational maturity. Organizations that PMI categorizes as "champions" (those with high project management maturity, effective governance, and aligned project portfolios with organizational strategy) achieve dramatically better results: higher percentages of projects completed on time and on budget, lower rates of scope creep, and higher rates of meeting business objectives. This suggests that the tools and practices for successful project management are known and available - the challenge is organizational adoption and consistent application.
PMI's surveys have specifically examined the relationship between risk management practices and project outcomes. The findings consistently show that organizations with mature risk management practices - including formal risk identification, quantitative risk analysis, risk response planning, and risk monitoring - have significantly better project outcomes than those without. The 2019 Pulse of the Profession report found that organizations with high risk management maturity completed 73% of projects meeting original goals, compared to 53% for organizations with low risk management maturity.
The PMI's Practice Standard for Project Risk Management and the PMBOK Guide (7th Edition, 2021) both recommend quantitative risk analysis, including Monte Carlo simulation, as a best practice for managing project uncertainty. However, PMI's surveys also show that many organizations have not adopted these practices: quantitative risk analysis remains uncommon in organizations with lower project management maturity. The gap between best practice and common practice represents a significant opportunity for improvement.
McKinsey & Company, in collaboration with the University of Oxford, published a study in 2012 examining the performance of 5,400 IT projects with budgets exceeding $15 million. The study, authored by Bloch, Blumberg, and Laartz and published in the McKinsey Quarterly under the title "Delivering Large-Scale IT Projects On Time, On Budget, and On Value," produced sobering findings:
The McKinsey study's finding that 17% of large IT projects threaten the existence of the company is particularly alarming. These "black swan" IT projects - the ones that go catastrophically wrong - are far more common than most organizations assume. The study identified several characteristics of these tail-risk projects, including ambitious scope, novel technology, multiple stakeholders with conflicting objectives, and weak project governance.
McKinsey's broader research on organizational decision-making, published in a 2019 survey of more than 1,200 global managers, found that respondents considered decision-making to be the most important driver of organizational performance - more important than portfolio management, resource allocation, or strategic planning. Yet only 20% of respondents reported that their organizations excelled at decision-making.
The survey also found that the quality of decision-making processes was strongly correlated with business outcomes. Organizations with high-quality decision processes (defined by criteria such as evidence-based analysis, transparent reasoning, appropriate stakeholder involvement, and timely execution) outperformed those with low-quality processes on both financial and non-financial metrics. These findings reinforce the case for structured decision intelligence practices and rigorous go/no-go frameworks.
Synthesizing the research from Kahneman, Flyvbjerg, the Standish Group, PMI, and McKinsey, we can identify a taxonomy of failure modes that account for the majority of plan failures. Understanding these modes is the first step toward addressing them.
As discussed in detail above, the planning fallacy is the systematic tendency to underestimate costs, overestimate benefits, and ignore the base rates of similar past projects. It is the most fundamental and pervasive failure mode, affecting virtually every plan and project to some degree. The planning fallacy operates unconsciously - even planners who are aware of the bias continue to produce optimistic estimates - and it is compounded by organizational incentives that reward optimism and punish conservatism.
How probabilistic planning addresses it: Monte Carlo simulation forces planners to specify ranges rather than single points, making the uncertainty visible. Reference class forecasting provides outside-view data to calibrate the ranges. The resulting probability distribution shows the actual likelihood of achieving the plan's targets, which is almost always lower than the planner initially assumed.
Traditional planning produces a single set of numbers: one revenue forecast, one cost estimate, one timeline. This single-point approach creates a false sense of precision, anchoring stakeholders to a specific outcome and preventing them from considering the range of alternatives. When the actual outcome differs from the plan (as it almost always does), the divergence is perceived as a failure, even if the actual outcome falls within a perfectly reasonable range of uncertainty.
The anchoring effect is particularly dangerous in budgeting and resource allocation. Once a budget number is established, it becomes the anchor for all subsequent discussions. Budget overruns are measured against the anchor, creating accountability for hitting a number that may have been unrealistically low from the beginning. This dynamic discourages conservative estimation (because higher estimates are harder to get approved) and rewards optimistic estimation (because the overrun, if it occurs, will be someone else's problem).
How probabilistic planning addresses it: Monte Carlo simulation replaces the single-point estimate with a probability distribution. Instead of a budget of $5 million, the analysis produces a P50 of $5 million (50% probability), a P80 of $6.2 million (80% probability of not exceeding this amount), and a P95 of $7.5 million. This range-based approach enables more realistic budgeting and contingency planning. The U.S. GAO recommends budgeting at the P80 level for this reason.
Scope creep - the uncontrolled expansion of project requirements after the project has begun - is one of the most commonly cited causes of project failure. The PMI Pulse of the Profession surveys consistently identify scope creep as a top factor in project failure. Scope creep occurs because stakeholders add requirements after the initial plan is approved, because the original requirements were inadequately defined, or because the team discovers additional work that was not anticipated in the original scope.
Scope creep is particularly insidious because each individual addition seems minor and reasonable. "Can we also add feature X? It's only a few days of work." But the cumulative effect of dozens of such additions can increase the project cost and timeline by 50% or more, with each addition justified as "too small to worry about" but the aggregate effect devastating to the plan.
How probabilistic planning addresses it: Sensitivity analysis identifies which scope elements have the greatest impact on the project outcome, focusing attention on the most consequential scope decisions. Monte Carlo simulation can include scope uncertainty as an explicit input variable, with the probability of additional scope items modeled as discrete events with associated cost and schedule impacts. This makes the cost of scope creep visible before it occurs.
Business plans often treat uncertain variables as independent when they are actually correlated. Revenue and costs may be correlated (both driven by market conditions). Activity durations may be correlated (if one task takes longer, related tasks probably will too, because they share the same technical or organizational challenges). Ignoring these correlations can dramatically underestimate the probability of extreme outcomes. If costs are correlated, the probability of multiple cost items simultaneously overrunning is much higher than if they were independent.
How probabilistic planning addresses it: Monte Carlo simulation can explicitly model correlations between input variables, ensuring that the simulation generates realistic combinations of values rather than artificial ones. The impact of correlations on the output distribution can be dramatic - adding positive correlations between cost items can increase the spread of the total cost distribution by 50% or more, revealing tail risks that an uncorrelated model would miss. For more on business risk analysis techniques, see our dedicated guide.
When a plan has been developed by a team over weeks or months, the team develops emotional and cognitive attachment to the plan. Confirmation bias causes them to seek and interpret information in ways that support the plan, while dismissing or discounting information that contradicts it. Groupthink - the tendency of cohesive groups to prioritize consensus over critical evaluation - further suppresses dissent. The result is a plan that has been evaluated primarily by its advocates, with critical information filtered out.
How probabilistic planning addresses it: Quantitative analysis provides an independent, objective assessment that is harder to dismiss than qualitative concerns. When a Monte Carlo simulation shows a 40% probability of the project losing money, it creates a concrete data point that must be addressed rather than a vague concern that can be waved away. Pre-mortem analysis and red-team reviews, conducted as part of the evidence-gathering process, further counteract groupthink.
Once a plan is underway and resources have been committed, the sunk cost fallacy creates pressure to continue even when new information suggests the plan is failing. The escalation of commitment - increasing investment in a losing course of action in the hope of recovering sunk costs - has been extensively studied in behavioral economics and organizational behavior. Barry Staw's seminal 1976 paper "Knee-Deep in the Big Muddy" documented the phenomenon, and subsequent research has confirmed that escalation of commitment is widespread in organizations.
How probabilistic planning addresses it: Predefined go/no-go decision points with explicit kill criteria provide an objective basis for stopping a failing project. The criteria, defined before the project begins, are immune to sunk cost bias because they focus on expected future performance rather than past investment. Regular Monte Carlo reassessments update the probability of success based on current information, providing an ongoing reality check.
Plans often fail because of risks that were never identified, let alone managed. The "unknown unknowns" - risks that the planning team did not even consider - are by definition invisible to the planning process. While it is impossible to identify all risks in advance, the research shows that most failed projects were undone by risks that could have been identified through systematic risk analysis, reference class analysis, or pre-mortem exercises. The risks were not truly "unknown" - they were simply not looked for.
How probabilistic planning addresses it: The process of defining input distributions for a Monte Carlo simulation forces systematic consideration of uncertainty in each project variable. The pre-mortem technique surfaces risks that might otherwise be overlooked. Reference class data reveals risks that affected similar past projects. And the simulation's output distribution - particularly its tails - reveals the aggregate impact of risk, even if individual risks are not fully identified.
IT projects have among the highest failure rates of any project category. The Standish Group data, McKinsey's research, and Flyvbjerg's expanded database all confirm this. The unique characteristics of IT projects that contribute to high failure rates include: rapidly changing technology (requirements can become obsolete during development), high complexity (software systems involve millions of interdependent components), difficulty of estimation (software development is a creative activity whose productivity varies enormously), and scope volatility (users often cannot articulate their needs until they see a working system).
The adoption of agile development methodologies has improved IT project outcomes, particularly for smaller projects, by reducing the planning horizon, incorporating frequent feedback, and adapting to changing requirements. However, agile methods do not eliminate the need for realistic estimation and risk management. A sprint estimate of "this feature will take three weeks" is subject to the same planning fallacy as a waterfall estimate of "this project will take 18 months" - just at a smaller scale.
Flyvbjerg's research focuses heavily on construction and infrastructure projects, which have a long and well-documented history of cost overruns and schedule delays. The median cost overrun for transportation infrastructure projects is approximately 28%, and for rail projects it is approximately 45%. Iconic projects such as the Sydney Opera House (original budget A$7 million, final cost A$102 million), the Channel Tunnel (original budget £4.65 billion, final cost approximately £9.5 billion), and Boston's Big Dig (original estimate $2.6 billion, final cost approximately $14.6 billion, including financing costs) illustrate the scale of potential overruns.
Construction projects are susceptible to several industry-specific failure modes: geotechnical surprises (underground conditions differ from expectations), regulatory changes during the project, design changes driven by stakeholder demands, weather and site access issues, labor disputes, and supply chain disruptions for specialized materials and equipment. These risks are well-known in the industry, yet they continue to be underestimated in project plans.
Pharmaceutical development has one of the highest failure rates of any business activity: approximately 85-90% of drug candidates that enter Phase I clinical trials never reach market approval, according to research by Thomas, Burns, Audette, and Carroll published in Clinical Pharmacology & Therapeutics (2016). The cost of developing a single approved drug is estimated at $1-2 billion, including the cost of the failed candidates (research by DiMasi, Grabowski, and Hansen published in the Journal of Health Economics, 2016).
The pharmaceutical industry's high failure rate is fundamentally different from the construction or IT failure patterns. In pharma, the failures are primarily driven by scientific uncertainty - will the drug work? will it be safe? - rather than by planning bias. However, the cost and timeline estimates for individual development programs are subject to the same planning fallacy as other industries. Clinical trials routinely take longer and cost more than planned, and the commercial value of approved drugs is often overestimated. Monte Carlo simulation is widely used in pharmaceutical portfolio management to model the combined uncertainty of technical success, development cost, development timeline, and commercial value.
Startup failure rates are high by design: startups are experiments in business model viability, and most experiments fail. Research from Harvard Business School's Shikhar Ghosh, published in the Wall Street Journal in 2012, found that approximately 75% of venture-backed startups fail to return investors' capital, and approximately 30-40% result in a total loss of the invested capital. The U.S. Bureau of Labor Statistics data shows that approximately 20% of new businesses fail in their first year and approximately 50% fail within five years.
Startup plans fail for reasons that overlap with but extend beyond the general failure modes. Market risk (the market may not want what the startup is building) is often the dominant uncertainty. Business model risk (the unit economics may not work) is another. Execution risk (the team may not be able to build what they have envisioned) compounds the other risks. And funding risk (the startup may run out of money before achieving product-market fit) creates a hard time constraint.
For startups, probabilistic planning through Monte Carlo simulation is particularly valuable for runway analysis: "Given our burn rate, revenue growth, and the uncertainty in both, what is the probability that we will run out of cash before reaching breakeven?" This question is existential for startups, and answering it probabilistically (rather than with a single-point projection) helps founders make better decisions about fundraising timing, spending priorities, and pivot vs. persevere choices.
The research on project failure also reveals what successful projects do differently. While no approach guarantees success, several practices are consistently associated with better outcomes across studies and industries.
Successful projects use realistic estimates that account for uncertainty, rather than optimistic estimates that assume everything will go according to plan. This does not mean padding estimates with arbitrary contingency; it means honestly assessing the range of possible outcomes and planning for the most likely case rather than the best case. Techniques such as reference class forecasting, three-point estimation, and Monte Carlo simulation support realistic planning by forcing planners to confront uncertainty rather than ignore it.
Flyvbjerg's research shows that modular projects - those composed of many small, repeatable units - dramatically outperform monolithic projects. Solar farms (composed of many identical panels), wind farms (composed of many identical turbines), and data centers (composed of many identical server racks) have among the lowest cost overruns of any project category. By contrast, one-of-a-kind megaprojects (Olympic venues, custom skyscrapers, bespoke IT systems) have among the highest.
The implication for business planning is clear: wherever possible, break large initiatives into smaller, independently deployable components. Launch a minimum viable product rather than a comprehensive system. Enter a new market with a pilot program before committing to a full rollout. Hire incrementally rather than in large batches. Each smaller unit has shorter planning horizons, faster feedback loops, and lower stakes, all of which contribute to more realistic planning and better outcomes.
Successful projects build in formal decision points - stage gates or go/no-go reviews - at which the project is reassessed based on current information. These reviews provide an opportunity to kill failing projects early, redirect resources to more promising initiatives, and update the plan based on what has been learned. Without these formal checkpoints, projects tend to drift forward on momentum, consuming increasing resources without periodic reality checks.
The PMI data consistently shows that organizations with mature risk management practices achieve better project outcomes. Active risk management means: identifying risks systematically (not just the obvious ones), quantifying risks where possible (using Monte Carlo simulation and other quantitative methods), developing response plans for the most significant risks, monitoring risk indicators throughout the project, and updating the risk assessment as new information becomes available.
One of the most puzzling findings in the research is that organizations do not seem to learn from their own experience. Flyvbjerg's finding that cost overrun patterns have not improved over decades is a stark illustration. The explanation is that learning requires systematic feedback, and most organizations do not have systematic processes for comparing planned outcomes to actual outcomes, identifying the sources of deviation, and incorporating the lessons into future planning. Post-project reviews, when they happen at all, are often cursory and focused on blame rather than learning.
Organizations that learn effectively from experience treat each project as a data point in a larger pattern. They maintain databases of actual vs. planned outcomes, analyze the systematic biases in their estimates, and adjust their planning processes accordingly. This is the organizational equivalent of calibration training: measuring the accuracy of estimates and using the results to improve future estimates. The calibration tracking feature in decision intelligence platforms supports this learning process by tracking how well predictions match actual outcomes over time.
The preceding sections have described the specific failure modes of traditional planning and, for each, noted how probabilistic planning provides a countermeasure. This section consolidates those insights into a comprehensive view of how a probabilistic approach transforms the planning process.
| Failure Mode | Traditional Planning | Probabilistic Planning |
|---|---|---|
| Planning fallacy | Inside-view estimates accepted at face value | Outside-view data calibrates ranges; simulation reveals true likelihood |
| Single-point anchoring | One budget, one timeline, one revenue forecast | Probability distributions with P10/P50/P80 percentiles |
| Ignored correlations | Variables treated as independent | Correlations modeled explicitly; tail risks revealed |
| Scope creep | Scope additions assessed informally | Each scope addition modeled with cost/schedule impact |
| Confirmation bias | Plan evaluated by advocates | Objective simulation results independent of advocacy |
| Sunk cost escalation | Projects continue on momentum | Predefined kill criteria; periodic re-simulation |
| Inadequate risk ID | Risk list based on brainstorming | Systematic uncertainty mapping; reference class data |
The transition from deterministic to probabilistic planning does not require abandoning traditional planning methods. The bottom-up estimate remains valuable as a starting point; the Monte Carlo simulation adds the uncertainty ranges that the bottom-up estimate omits. The project schedule remains necessary for sequencing and resource allocation; the simulation adds the probabilistic assessment of schedule risk. The business case remains the central document for decision-making; the simulation enriches the business case with probability distributions rather than point estimates.
What the transition does require is a cultural shift: an organizational willingness to acknowledge and quantify uncertainty rather than hiding behind the false precision of single-point estimates. This shift is uncomfortable because it makes uncertainty visible, and visible uncertainty can be unsettling. But the uncertainty exists whether or not it is acknowledged. Plans based on single-point estimates do not reduce uncertainty; they merely hide it. And hidden uncertainty, as decades of research have demonstrated, is far more dangerous than visible uncertainty.
Modern platforms like Incertive make this transition practical by providing intuitive tools for probabilistic planning that do not require statistical expertise. The barrier to honest, quantitative uncertainty assessment has never been lower. What remains is the organizational will to embrace uncertainty rather than deny it. For additional perspective, explore our guide on the reasons why project plans fail.
The failure rate depends on how "failure" is defined and what type of plan is measured. For IT projects, the Standish Group CHAOS reports have consistently found that only about 30-35% of projects are completed on time, on budget, and with satisfactory results. For large infrastructure projects, Bent Flyvbjerg's research shows that approximately 90% experience cost overruns. For startups, research from Harvard Business School's Shikhar Ghosh suggests that about 75% of venture-backed startups fail to return investors' capital. The common thread is that the majority of plans, across all domains, fail to meet their original targets.
The planning fallacy, identified by Daniel Kahneman and Amos Tversky in 1979, is the systematic tendency to underestimate the time, cost, and risk of planned actions while overestimating their benefits. It occurs because planners focus on the specific plan (the "inside view") rather than on the base rates of similar past plans (the "outside view"). The planning fallacy is not simple optimism - it is a specific cognitive bias that affects even experienced planners who are aware of the bias. It is one of the primary reasons that business plans consistently underperform their projections.
Experience does not automatically correct the planning fallacy because the bias operates at the level of the planning process, not individual knowledge. Experienced planners still construct bottom-up estimates by imagining how each step will go (the inside view), which naturally focuses on the plan for success rather than the many ways things can go wrong. Additionally, organizational incentives often reward optimistic estimates: projects with conservative estimates are less likely to be funded, and managers who give optimistic timelines appear more capable. Flyvbjerg's research shows that cost overrun patterns have remained stable over decades, suggesting that neither experience nor technology improvements have reduced the bias.
Optimism bias is a general tendency to overestimate the likelihood of positive events and underestimate the likelihood of negative events. The planning fallacy is a specific manifestation of optimism bias applied to plans and projects: the tendency to underestimate time, cost, and risk while overestimating benefits. The planning fallacy also involves a specific cognitive mechanism - reliance on the inside view (the specific plan) rather than the outside view (base rates of similar plans) - that goes beyond general optimism. Additionally, the planning fallacy can be amplified by strategic misrepresentation, where planners deliberately underestimate costs to get projects approved.
Monte Carlo simulation addresses several root causes of plan failure simultaneously. It forces planners to specify ranges rather than single points, counteracting false precision. It reveals the compounding effect of multiple uncertainties, which intuition consistently underestimates. It produces probability distributions rather than point estimates, making the likelihood of different outcomes visible. And the sensitivity analysis identifies which variables matter most, focusing attention on the key risks. The probabilistic output makes it much harder to maintain unrealistic expectations because the range of possible outcomes is explicitly visible.
Reference class forecasting is a technique for improving estimate accuracy by anchoring to the outcomes of similar past projects rather than relying solely on bottom-up analysis of the specific project. The approach involves three steps: (1) identify a reference class of comparable past projects, (2) establish the distribution of outcomes in that reference class, and (3) position the specific project within that distribution. For example, if historical data shows that 70% of similar IT projects experienced cost overruns of 20% or more, a new IT project should incorporate that base rate into its estimates rather than assuming it will be different.
Research from multiple sources converges on a consistent set of failure modes: unrealistic cost and schedule estimates (planning fallacy), scope creep (uncontrolled expansion of project requirements), inadequate risk management (failure to identify and mitigate key risks), poor stakeholder management (misaligned expectations and inadequate communication), resource constraints (insufficient or poorly allocated resources), and lack of executive support. The PMI Pulse of the Profession surveys consistently identify these factors as the top causes of project failure across industries.
No. Probabilistic planning cannot eliminate uncertainty or guarantee success. What it can do is provide a more realistic assessment of the range of possible outcomes, identify the key risks that need to be managed, establish appropriate contingency levels, and enable better-informed go/no-go decisions. A project planned probabilistically may still fail, but the probability of failure will be honestly assessed and communicated, and the contingency plans will be appropriate for the actual level of risk. The goal is not to eliminate failure but to make better decisions about which projects to pursue and how to manage them.
The Standish Group's CHAOS reports, published since 1994, track the outcomes of IT projects. The reports consistently find that only about one-third of IT projects are completed on time, on budget, and with satisfactory results (categorized as "successful"). Roughly half are "challenged" (completed but over budget, over time, or with fewer features than planned), and about 15-20% are "failed" (cancelled or never used). While the specific percentages have varied somewhat across editions, the overall pattern - that the majority of IT projects do not meet their original targets - has been remarkably consistent over three decades.
Five practices will make your business plan significantly more realistic: (1) Use ranges instead of single-point estimates for all uncertain variables. (2) Apply reference class data - research how similar projects or businesses have actually performed, not just how they were planned to perform. (3) Run a Monte Carlo simulation to see the full distribution of possible outcomes. (4) Conduct a pre-mortem: imagine the plan has failed and identify the most plausible causes. (5) Use sensitivity analysis to identify the variables that matter most, and invest your planning effort in reducing uncertainty about those variables rather than refining estimates that do not significantly affect the outcome.
The research on why business plans fail tells a consistent story across decades, disciplines, and geographies. Plans fail primarily because of systematic cognitive biases - the planning fallacy, optimism bias, anchoring, confirmation bias - that cause planners to underestimate costs, overestimate benefits, and ignore the base rates of similar past projects. These biases are compounded by organizational incentives that reward optimism, groupthink that suppresses dissent, and decision processes that lack rigor and accountability.
The good news is that the countermeasures are well-understood. Reference class forecasting provides the outside view that Kahneman prescribed. Monte Carlo simulation quantifies uncertainty and reveals the true range of possible outcomes. Sensitivity analysis identifies the variables that matter most. Go/no-go frameworks provide structured decision points that prevent failing projects from consuming resources indefinitely. Calibration training improves the accuracy of expert estimates over time.
The bad news is that adoption of these countermeasures remains low. Despite decades of evidence, most organizations still plan with single-point estimates, still use the inside view, still lack formal go/no-go processes, and still do not learn systematically from their planning failures. The gap between best practice and common practice is enormous, and closing it represents one of the highest-return investments any organization can make.
The choice is not between certain success and planning failure. Uncertainty is irreducible, and even the best plans will sometimes fail. The choice is between honest, quantitative engagement with uncertainty - which leads to realistic plans, appropriate contingencies, and well-informed decisions - and the denial of uncertainty through single-point estimates, which leads to unrealistic expectations, inadequate contingencies, and surprised, disappointed stakeholders. The research is clear about which approach produces better outcomes.
Incertive makes probabilistic planning accessible. Replace single-point estimates with probability distributions, identify the variables that matter most, and make decisions with a clear understanding of the risks.
Get StartedBack to Blog