AI Cost Estimation: What a Federal Guide Demands of Any Model

A trained model is judged like any estimate: comprehensive, documented, accurate, credible. What GAO-20-195G and a 28-study review say.

Guide: AI Tools and IndustryLast verified: October 20265 min read

An AI cost estimate is a statistical estimate of a project's cost produced by a machine learning model trained on data from past projects. The standard it must meet is the same one applied to any estimate: the U.S. Government Accountability Office (GAO) says a reliable estimate is comprehensive, well documented, accurate, and credible. Research reviews suggest machine learning can improve accuracy in software effort estimation, but the evidence is tied to the datasets used.

Key takeaways

  • The GAO Cost Estimating and Assessment Guide (GAO-20-195G, March 2020) sets four characteristics of a reliable estimate and a 12-step process; the guide does not mention machine learning.
  • The guide states that reliable estimates are rooted in historical data, which is also the raw material a machine learning model needs.
  • A systematic review of 28 studies found that ensemble machine learning methods generally achieved better accuracy than single techniques for software effort estimation.
  • A single number is not enough: the guide's own example shows a point estimate of USD 2.47 billion rising to USD 2.79 billion at an 80 percent confidence level.

What counts as a reliable cost estimate

The GAO guide, published in March 2020, describes four characteristics of a high-quality estimate. A comprehensive estimate includes all possible costs and ensures nothing is omitted or double-counted. A well-documented estimate can be repeated or updated and traced to original sources. An accurate estimate uses the best method for each cost element, adjusted for inflation, with few mistakes. A credible estimate includes sensitivity, risk, and uncertainty analysis, is reconciled with an independent estimate, and is cross-checked with alternate methods.

These tests do not depend on how the number was produced. A spreadsheet, an expert panel, and a trained model are all judged against the same four characteristics, which makes them a practical yardstick for any vendor claim about "AI-powered" estimating.

Where machine learning fits in the guide's methods

The guide names three commonly used methods. An analogy uses the cost of a similar program and adjusts for differences. Engineering build-up estimates each piece at the lowest level of the work breakdown structure and sums them. The parametric method relates cost to technical, performance, or program parameters through a statistical relationship. A model trained on past project records to predict cost from project features resembles the parametric method, though that reading is an interpretation, not a statement in the guide.

The guide adds that parametric models are useful for cross-checking an estimate derived by other means, and that their inputs should be objectively assessed and tailored to the product being estimated. It also notes that access to the underlying data of parametric models may be limited when the models are proprietary. The same concern applies to a commercial model whose training data cannot be inspected.

Why historical data decides the outcome

On data, the guide is direct: the quality of the data affects the estimate's overall credibility, and depending on that quality, an estimate can range from a rough guess to a highly defensible cost position. Estimators usually build estimates for new programs from existing ones and adjust for differences.

A machine learning model works the same way at larger scale. It learns from records of past projects, so records that are incomplete, inconsistent, or unlike the new project limit what it can predict. A 2023 review of the field by Victor Uc-Cetina observes that many software companies now recognize that storing project data is valuable for training prediction models.

What research reviews report on accuracy

For software projects, the most studied use of machine learning in estimation is predicting development effort. A systematic review by Yasir Mahmood and colleagues, posted to arXiv in January 2021, evaluated 28 studies, 14 using ensemble methods and 14 using single techniques, with two accuracy measures: mean magnitude of relative error and PRED(25). The authors found that machine learning is the most common basis for ensemble estimation techniques and that ensembles generally reached better accuracy than single techniques.

That finding is narrow. It compares machine learning techniques with each other on published software datasets. It does not show that a trained model beats an experienced estimator on a bridge, a hospital, or a satellite, and an arXiv posting is not the same as a peer-reviewed meta-analysis across industries.

A single number hides the range

The guide's risk chapter uses a worked example. A program had a point estimate of USD 2.47 billion. After a Monte Carlo simulation, which randomly samples cost values thousands of times, the cost at the 50 percent confidence level was USD 2.61 billion. At an 80 percent confidence level the figure was USD 2.79 billion, about USD 320 million above the point estimate.

Measure in the GAO exampleCost (USD)
Point estimate2.47 billion
50 percent confidence level2.61 billion
80 percent confidence level2.79 billion

A model that outputs one figure without a range omits what the guide treats as central to credibility.

Questions to put to an AI cost estimate

  • Comprehensive: does the model cover every cost element, or only the ones present in its training records?
  • Well documented: can the data sources and assumptions be traced?
  • Accurate: how was error measured, and on projects the model had not seen?
  • Credible: is there a range, a sensitivity test, and a cross-check against another method?
  • Updated: is the estimate revised with actual costs as the project proceeds, as the guide's twelfth step requires?

FAQ

Can AI replace a human cost estimator?

The sources reviewed do not show that. The GAO guide treats estimating as a documented process with independent review, and the machine learning research reviewed here compares techniques with one another rather than with professional estimators.

What data does an AI cost model need?

Records of past projects with costs and the features that drive them. The GAO guide stresses that valid, applicable historical data is a key step in any sound estimate.

Is a more complex model more accurate?

Not necessarily. The review of 28 studies found ensembles generally more accurate than single techniques, but results depend on the datasets used and the accuracy measures chosen.

Does the GAO guide apply to private companies?

The guide is written for federal programs. Private firms can use its four characteristics as a checklist, though it carries no legal force for them.

Last verified: October 2026

Sources

  • Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Program Costs, GAO-20-195G, U.S. Government Accountability Office, March 2020
  • Software Effort Estimation Accuracy Prediction of Machine Learning Techniques: A Systematic Performance Evaluation, Mahmood Y et al., arXiv:2101.10658, 2021
  • Recent Advances in Software Effort Estimation using Machine Learning, Uc-Cetina V, arXiv:2303.03482, 2023

New to artificial intelligence?

Start with the basics, then follow the guides to tools, rules, and effects on society.

Start with AI Basics