Back to blog

Software Estimation: Why Project Forecasts Miss and How to Give Your Board a Date You Can Defend

Software Estimation: Why Project Forecasts Miss and How to Give Your Board a Date You Can Defend
<p>A CTO tells the board that a project will ship in four months. At month five, the team reports it is "90% done." At month seven, the board stops asking. This pattern is so common that most leaders treat it as weather rather than a process failure. At AEGONTECH LLC, we have spent years building and rescuing software projects, and we have learned that missed deadlines are rarely caused by slow engineers. They are caused by how the original estimate was produced, communicated, and defended.</p> <h2>Key Takeaways</h2> <ul> <li>An estimate is a probability distribution, not a date. A single number hides the uncertainty that matters most.</li> <li>Estimates fail mainly from unknown work, optimism bias, and pressure to anchor to a desired answer, not from poor coding speed.</li> <li>Forecasting from your team's real throughput beats guessing hours for individual tasks.</li> <li>Re-estimating at fixed checkpoints protects the budget far better than defending the first number.</li> <li>The estimation process you choose should match the contract model and the level of uncertainty in the project.</li> </ul> <h2>Why Do Software Estimates Keep Failing?</h2> <p>Software estimates fail because they are produced at the moment of maximum ignorance and then treated as commitments. Early in a project, the team knows the least about the domain, the integrations, and the edge cases, yet this is when the budget and the launch date get fixed.</p> <p>The research is consistent. The Standish Group's long-running CHAOS reports have found that only around 30% of software projects are delivered on time, on budget, and with the intended scope. Bent Flyvbjerg's analysis of large IT projects found that roughly one in six overran its budget by an average of 200% and its schedule by almost 70%. The overruns are not random noise. They are systematically in one direction, which tells us the cause is structural.</p> <p>Three forces drive that structure. First, <strong>unknown unknowns</strong>: work nobody thought to list, such as data migration, permission models, and third-party API quirks. Second, <strong>optimism bias</strong>: people estimate the happy path and forget that reviews, bug fixes, deployments, and meetings consume real time. Third, <strong>anchoring</strong>: when a stakeholder says "we need this by June," the estimate quietly bends toward June.</p> <p>A quotable rule we use internally: <em>an estimate that cannot be wrong is not an estimate, it is a wish.</em></p>

Inline blog image 1

<h2>What Is the Difference Between an Estimate, a Target, and a Commitment?</h2> <p>An estimate is a prediction of effort based on current knowledge, a target is a business goal, and a commitment is a promise with consequences. Confusing them is the most expensive mistake in software planning.</p> <p>When a sales team's target becomes the engineering team's "estimate," the number stops carrying information. The team has not learned that the work is smaller; it has learned that disagreement is unwelcome. A healthy process keeps the three separate. Engineering produces a range. Product and business leaders choose a target inside or outside that range with open eyes. A commitment is made only to the part of the scope that the evidence supports.</p> <p>This is also where the choice of contract model matters. A fixed-price contract forces a single number up front, so the vendor either pads heavily or absorbs risk and cuts corners. A time-and-materials or dedicated-team model lets the estimate stay a living forecast. We have written before about how that contract decision decides who carries project risk, and estimation is the mechanism through which that risk becomes visible.</p> <h2>How Does the Cone of Uncertainty Change Your Planning?</h2> <p>The cone of uncertainty describes how estimate accuracy improves as a project progresses. At the initial concept stage, a good estimate can be off by a factor of 4x in either direction, from 0.25x to 4x of the eventual actual effort. After detailed requirements, the range narrows to roughly 0.67x to 1.5x. Only late in the build does it approach the true figure.</p> <p>The practical implication is blunt: if someone gives you a single-point estimate in week one, they are either guessing or hiding the range. Ask for a range and for the assumptions behind it. The assumptions are more valuable than the number, because each one is a risk you can retire with a spike (a short, time-boxed investigation) before committing budget.</p> <h2>Story Points, Hours, or Throughput: Which Forecasting Method Works Best?</h2> <p>Throughput-based forecasting generally works best because it relies on what the team has actually delivered rather than on what individuals believe they can deliver. Hour estimates and story points both ask humans to predict effort, which is exactly the thing humans are worst at.</p> <p>Here is how the three approaches compare in practice:</p> <ul> <li><strong>Hour estimates:</strong> precise-looking, rarely accurate. Fine for tiny, well-understood tasks; unreliable for anything with integration or discovery work.</li> <li><strong>Story points:</strong> a relative-size measure used in Agile and Scrum teams. Useful for conversation, but they drift over time and get misused as productivity metrics.</li> <li><strong>Throughput and cycle time:</strong> count finished items per week and measure how long items take from start to done. Use those historical numbers to forecast.</li> </ul> <p>The strongest technique we apply is a <strong>Monte Carlo simulation</strong>: a method that reruns the delivery forecast thousands of times using the team's real historical throughput, then reports results as probabilities such as "85% likely to finish by week 14, 50% likely by week 11." It replaces a false-precision date with an honest confidence level, and stakeholders quickly learn to read it. Teams that track the four DORA metrics already collect most of the data required, as we covered in our piece on measuring engineering performance.</p>

Inline blog image 2

<h2>How Do You Estimate Work You Have Never Done Before?</h2> <p>You estimate novel work by shrinking the unknown first and sizing second. Run a time-boxed spike, build a thin vertical slice through the riskiest part of the system, and use what you learn to re-estimate. Spending 5% of the budget to remove the largest uncertainty routinely saves far more than that downstream.</p> <p>At AEGONTECH, the same principle shaped our own products. When we built Dolfy.ai and Mimicall.app, the riskiest assumptions were not the user interfaces. They were third-party API behavior, media processing cost, and latency under real traffic. We validated those first with small prototypes, and the later estimates were tighter because the biggest unknowns had already been converted into known quantities. For Maximus IPTV Player, the equivalent risk was playback behavior across devices, so device testing came before feature planning.</p> <p>Another reliable tactic is <strong>reference class forecasting</strong>: instead of reasoning from the details of this project, ask how long similar projects actually took. Daniel Kahneman called this taking the "outside view," and it consistently beats bottom-up optimism. If your last three integrations with payment providers took six to nine weeks, a four-week estimate needs extraordinary justification.</p> <h2>What Buffers and Checkpoints Protect the Budget?</h2> <p>Explicit, visible buffers and fixed re-estimation checkpoints protect the budget better than padded task estimates. Hidden padding gets consumed by Parkinson's Law, the observation that work expands to fill the time available, while an explicit project-level buffer can be spent deliberately on real surprises.</p> <p>We recommend a simple rhythm for any project longer than eight weeks:</p> <ul> <li>Break scope into must-have, should-have, and could-have tiers, and commit only to the first tier.</li> <li>Add a visible contingency of 20-30% for projects with significant integration or legacy-system risk.</li> <li>Re-forecast every two to four weeks using actual throughput, and share the updated range with stakeholders immediately.</li> <li>When the forecast moves, negotiate scope or date at that moment rather than at the deadline.</li> </ul> <p>The point of checkpoints is to make bad news cheap. A two-week slip announced in week six is a scheduling conversation. The same slip discovered in week twenty is a crisis. Good engineering leadership makes early honesty the safest behavior on the team.</p> <h2>How Should Decision-Makers Evaluate a Vendor's Estimate?</h2> <p>Evaluate a vendor's estimate by looking at its structure rather than its total. A credible estimate comes as a range, lists its assumptions, separates discovery from delivery, identifies the top risks, and explains how it will be updated. A suspiciously low and precise number is a warning sign, not a bargain.</p> <p>Questions worth asking any software partner, including AEGONTECH, include: What would make this estimate wrong? Which parts are you least sure about? How do you forecast from actual progress? What happens to scope and price when an assumption fails? Vendors who answer fluently have done this before. Vendors who deflect are usually relying on the contract to absorb their uncertainty, and you will pay for it eventually, either in change requests or in quality.</p> <p>Also watch for what the estimate leaves out. Testing, security review (aligned with OWASP guidance), CI/CD pipeline setup, documentation, deployment, and post-launch stabilization are frequently missing from low bids. Leaving them out does not remove the work; it moves the cost to a later, less convenient moment.</p> <h2>Frequently Asked Questions</h2> <h3>Can software estimates ever be accurate?</h3> <p>Yes, within a range, and accuracy improves as uncertainty falls. Teams with stable membership, small batch sizes, and several months of throughput data can forecast delivery with useful confidence. Accuracy to the day on novel work is not realistic, and any process that promises it is hiding risk.</p> <h3>Should we stop estimating and just work iteratively?</h3> <p>No. Even fully iterative teams need forecasts for budgeting, hiring, and launch planning. The shift is from estimating every task in advance to forecasting from real delivery data and refreshing the forecast regularly. Lightweight estimation plus frequent re-forecasting gives most of the benefit with a fraction of the overhead.</p> <h3>How much contingency should we add to a software project budget?</h3> <p>For well-understood work with an experienced team, 10-15% is often enough. For projects involving legacy systems, new integrations, or unclear requirements, 20-30% is more realistic. The key is to make the contingency visible and to tie its release to specific, named risks rather than letting it dissolve into task padding.</p> <h3>What is the most common cause of estimate overruns?</h3> <p>Scope growth and unlisted work, not slow coding. Undiscovered requirements, integration surprises, and underestimated testing and deployment effort account for most overruns. Reducing them requires early discovery work and an explicit assumptions list, not asking developers to work faster.</p> <h2>Conclusion: Treat Estimates as Forecasts You Keep Earning</h2> <p>The teams that deliver predictably are not better at guessing. They separate estimates from targets, express uncertainty honestly, forecast from real throughput, and re-plan early when reality disagrees. That discipline turns the "90% done" problem into a routine, visible adjustment instead of a late surprise.</p> <p>If you are planning a new product, a modernization effort, or evaluating a development partner and want an estimate you can actually defend to your board, talk to the team at <a href="https://www.aegontech.dev">AEGONTECH LLC</a>. We are glad to review your scope, name the assumptions that carry the most risk, and give you a range you can plan around.</p>