SN L4b — Estimation

A revision summary of the second Scope lecture. An estimate is “the information we use to evaluate the cost of a future decision” [1], so every number in the lecture answers two questions: how much, and how sure are you? The running example is Workopolis, a job board with fifteen user stories for job seekers, recruiters and the site owner. Each section links to the handbook page carrying the detail and the sources.

1. Why estimates go wrong

A review of ten surveys puts the typical cost overrun between 30 and 40%; Standish’s famous 89% is the outlier [2], [3]. So a 1,000-hour estimate means 1,300 to 1,400 hours expected: carry the difference, or say why you will beat the average. And before believing any overrun figure, ask what it was measured against [4].

Three numbers have three owners. The estimate is what you believe, the target is what they want, the commitment is what you promise, the estimate plus contingency. The estimate feeds the budget; it is not the budget [5]. “Do not mix estimation of most likely effort with planning, budgeting or pricing” [4].

An early estimate is a range. At the initial concept, the real effort can be anywhere from 0.25 to 4 times the most likely number, a spread of 16 to 1; after requirements it is still about ±50%, and it narrows only as the work is done [6]. Give every estimate as a range, say the phase, and name the milestone of the next one.

When the budget comes first, the estimate’s job changes. You are given 600 hours to build Workopolis with Scrum: call 600 the budget, estimate independently at six effective hours a day, and if the estimate is bigger, cut stories in the product owner’s order. You can “negotiate the inputs to the estimate … but not the estimate itself” [6].

Exam tip: given a sponsor’s number, first say which of the three it is. A deadline is a target.

→ Estimates · Estimation approaches

2. Sizing the product

Scope becomes size, size becomes effort, effort becomes schedule. The size is the box everything hangs from: “the most significant input to the COCOMO II model is Size” [7].

Lines of code are a poor size for an estimate. They do not exist when the estimate is due, they differ for the same function in another language, and even after the fact a report of “163,000 source code instructions” can be misread by a factor of three [8]. Function points count what the user gets (inputs, outputs, inquiries and files), from the requirements, and the same screens score the same in C++ or in Clarion [9].

Two questions classify every transaction. Does it change a stored file? Then it is an external input (EI): “The only component that maintains an internal logical file is an external input” [9]. If not, does it derive data? Then an external output (EO); otherwise an external inquiry (EQ). For example, on Workopolis Upload a resume writes a file, so it is an EI; Search job announcements only retrieves, so it is an EQ. For files, count kinds of subgroup, never records: a department holding 20 employees has two record types [10].

The count is reliable; an estimate from it is not. Two trained counters of the same system differ by a median of 12% [11]. Albrecht’s line from function points to effort fits 24 IBM projects closely, yet its typical error on one project is about 85%, and he advised that such formula estimates “be used only to validate and provide perspective on primary estimates” [12].

→ Function Point Analysis · FPA explained

3. Expert judgment

Miranda groups estimation methods into six families: judgment, engineering (a cost function built from what drives the work), counting, industry norms, analogy and parametric models [5]. What separates them is the last step: “the quantification step—that is, the final step that transforms the input into the effort estimate” [13]. A person takes it, or a formula does.

An expert works top-down, against finished projects, or bottom-up, summing the WBS activities. Across seven teams, bottom-up was off by 23% and top-down by 45%, but the two top-down teams that recalled a very similar project with known effort were among the most accurate, at 12%: “the recall of very similar previously completed projects seemed to be a pre-condition for accurate top-down strategy based estimates” [14]. Know a similar project? Go top-down. If not, decompose.

Judgment has known biases. In one experiment, 23 people read the same brief, with one sentence from a client’s manager: “I admit I have no experience with software projects, but I guess this will take about 2 months to finish.” With a 20-month anchor the mean estimate was 17.4 months, with none 8.3, with 2 months 6.8 [15]. So nobody, the sponsor included, says a number before the estimators have written theirs.

Each bias has a fix that changes the input. Availability: an activity checklist and searchable history. Anchoring: strip numbers from the brief. Representativeness (“it’s just like the last one”): many past tasks, not the closest one. Confirmation: an independent second estimate. All four feed overconfidence; professionals’ 90% ranges held the actual only 35% of the time [16]. Warning people does not remove a bias; changing what they see does.

→ Expert judgment

4. Estimating as a group

Planning Poker gives each estimator a card to pick privately; all cards turn over together; if they differ, the highest and lowest explain, and the team votes again. With no consensus: “Defer the story, split it, or take the low estimate.” Grenning’s deck is 1, 2, 3, 5, 7, 10 and ∞, where ∞ means split it, and his purpose is plain: “Precision of individual estimates is not the goal. Determining the project scope is” [17]. No averages, no numbers off the deck [18].

Eduardo’s Variation, this course’s version, adds four rules [5]: agree a medium story as the baseline first; discuss coincidences too, since the same card can rest on different reasons; play one more round and record every estimator’s final card; and after all stories, probe unusual disagreement rather than averaging blindly. The rules are untested course practice.

A uniform-looking backlog is not. One senior developer sized all fifteen Workopolis stories; with Search jobs at 5, Register came out 14 times Log out, and the same expert gave Register 14.4 in one session and 13.0 in the next [19].

Poker removes the first speaker’s anchor and brings in the quiet members, but not the first round’s: “We found no story having the final planning poker estimate outside these boundaries” [20]. Matching cards end the talk even when the reasons differ, and on unfamiliar stories Poker did worse than plain discussion [21]. If the talk reveals new work, vote again from zero.

Story points are easy to agree on, because people compare better than they measure, and hard to compare: a point means something in one team against one reference, becomes a date only through velocity, and inflates once velocity is rewarded [18]. On ten Workopolis stories, comparing each with all the others gave a project total of 44.6; comparing each with only two gave 99.7 [19]. Compare each story with at least four others.

Paired comparisons ask only how much bigger is X than Y? and then check how consistent the answers are [22]. In one example, fixing a single bad judgment took the inconsistency index from 0.94 to 0.27, under the acceptable 0.35 [19]. For an architecture item unlike the rest of the backlog, compare it with an item of known size, or estimate it bottom-up.

Exam tip: “Poker prevents anchoring” is half right. It removes the first speaker’s anchor; the first round’s cards then fence the result.

→ Planning Poker · Paired comparison

5. Parametric models

A parametric model is a regression of effort on size, of the form E = A + B × size^C, with the constants fitted to somebody else’s past projects [23].

COCOMO II fits A = 2.94 and B = 0.91 to 161 projects, with seventeen effort multipliers once the architecture exists and seven before [7]. Its exponent is usually above one: “If E > 1.0, the project exhibits diseconomies of scale”, from growing communication and integration overhead [7]. Run it with its drivers: without them COCOMO landed within 1.3 times the actuals 29% of the time, with them 78% [13].

Three of the five scale factors are a manager’s choice: architecture and risk resolution, team cohesion and process maturity. The same 100 KSLOC product costs 194 person-months with every factor at its best and 832 at its worst. Compressing the schedule to 75% of nominal costs 43% more effort, while “Stretch-outs do not add or decrease effort”. And integration and test take 19% of the effort on a small project but 31% on a large one [7].

Calibrate before you trust a model. On one CAD system from Pressman’s textbook, the seven published models his chapter lists give 42 to 323 person-months; “Estimation models must be calibrated for local needs!” [23]. Five of your own finished projects are enough to refit the constant A [7]; until then, a model is a cross-check.

→ COCOMO · COCOMO II drivers

6. Combining methods

The 2009 debate sets the two sides out. Jørgensen: an expert can use what no model takes in, “a serious conflict between the project leader Peter and his best programmer Paul”, and a model’s inputs are judgments anyway. Boehm: “A major advantage of a parametric model is that it doesn’t modify its estimates when customers, managers, or marketers apply pressure” [13]. Across fifteen studies, five favoured experts, five models and five found no difference [24]. Use a calibrated model to check the expert, not to replace them.

Rules of thumb are checks, not estimates. Capers Jones’s rules turn function points into schedule (FP0.4 months), team size (FP ÷ 150) and defects (FP1.25), and he warns that they “should not be used for contracts, bids, or other serious business purposes” [25]. His article’s rules and his book’s table disagree by 2.4 times on the same 100,000 lines of C [5], so ask which hours and activities a norm counts.

Combine independent estimates, then read the spread. Twenty professionals estimating one real project alone averaged 1,088 hours; in five mixed groups, 1,548; the project took 2,365, and one expert alone had said 220. The groups “sometimes even identified necessary activities that none of the individuals had reported” [26]. A wide spread usually means the scope was misread or the productivity data is wrong [23]. With no history, as for L4a’s invented Quill: three people estimate alone, top-down from a norm and bottom-up over the WBS, check with a rule, report a range, and start recording actuals.

→ Estimation approaches · Expert judgment

Practice questions

1. Why should an estimator never hear the budget before estimating, and what should happen when the budget is fixed first?

Answer

A budget is a target, and a number heard first anchors the estimate [15]. Estimate independently, then treat the gap as a scope decision for the product owner [6].

2. Why is a function-point count more trustworthy than an effort estimate made from it?

Answer

Trained counters differ by a median of 12% [11]. The step from size to effort leaves out the team, the technology and hidden complexity, so on one project Albrecht’s formula typically missed by about 85% [12].

3. Compare top-down and bottom-up expert estimation. When would you choose each?

Answer

Top-down works only when the estimator recalls a very similar project and its actual effort; bottom-up forces an understanding of the work but loses whatever activity was forgotten [14]. Without a close analogue, decompose, and protect the sum with a checklist.

4. What does Planning Poker fix, and what does it leave unfixed?

Answer

Private cards remove the first speaker’s anchor and bring in the quiet members. The first round still fences the final estimate [20], and matching cards end the discussion even when the reasons differ, which Eduardo’s Variation targets by discussing coincidences [5].

5. “Our team tried COCOMO and it was useless.” What would you ask before agreeing?

Answer

Was it calibrated on our own projects, and run with its cost drivers? Without drivers it landed within 1.3 times the actuals 29% of the time; with them, 78% [13]. An uncalibrated model is a cross-check at best.

Where this goes next

Workopolis has a range; the sponsor still wants a date. Do you commit to the whole scope up front, or one sprint at a time? That is the next lecture.

Acknowledgments

This content is heavily inspired by and adapted from lectures by Eduardo Miranda and David Root [27] on software project management. The commitment-level planning picture, the six families, the 600-hour exercise, the Workopolis stories and Eduardo’s Variation of Planning Poker are Miranda’s [5].

References

  1. C. L. Hoover, M. Rosso-Llopart, and G. Taran,
    Evaluating Project Decisions: Case Studies in Software Engineering. Addison-Wesley Professional, 2010. [Online]. Available: https://www.oreilly.com/library/view/evaluating-project-decisions/9780321685629/
  2. K. Moløkken and M. Jørgensen,
    “A Review of Surveys on Software Effort Estimation,” in Proceedings of the 2003 International Symposium on Empirical Software Engineering (ISESE 2003), IEEE Computer Society, 2003, pp. 223–230. doi: 10.1109/ISESE.2003.1237981.
  3. J. L. Eveleens and C. Verhoef,
    “The Rise and Fall of the Chaos Report Figures,” IEEE Software, vol. 27, no. 1, pp. 30–36, 2010, doi: 10.1109/MS.2009.154.
  4. S. Grimstad, M. Jørgensen, and K. Moløkken-Østvold,
    “Software Effort Estimation Terminology: The Tower of Babel,” Information and Software Technology, vol. 48, no. 4, pp. 302–310, Apr. 2006, doi: 10.1016/j.infsof.2005.04.004.
  5. E. Miranda,
    “Managing Software Development.” 2014.
  6. S. McConnell,
    Rapid development: taming wild software schedules. Pearson Education, 1996.
  7. Center for Software Engineering, USC,
    COCOMO II Model Definition Manual. Center for Software Engineering, University of Southern California, 2000.
  8. R. E. Park,
    “Software Size Measurement: A Framework for Counting Source Statements,” Software Engineering Institute, Carnegie Mellon University, Technical Report CMU/SEI-92-TR-020, Sept. 1992. doi: 10.21236/ADA258304.
  9. D. Longstreet,
    Function Points Analysis Training Course. Longstreet Consulting Inc.
  10. N. Gunti,
    “Function Point Analysis Examined,” Avenue A | Razorfish, White paper, Sept. 2006.
  11. C. F. Kemerer,
    “Reliability of Function Points Measurement: A Field Experiment,” Communications of the ACM, vol. 36, no. 2, pp. 85–97, Feb. 1993, doi: 10.1145/151220.151230.
  12. A. J. Albrecht and J. E. Gaffney Jr.,
    “Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation,” IEEE Transactions on Software Engineering, vol. SE-9, no. 6, pp. 639–648, Nov. 1983, doi: 10.1109/TSE.1983.235271.
  13. M. Jørgensen, B. Boehm, and S. Rifkin,
    “Software Development Effort Estimation: Formal Models or Expert Judgment?,” IEEE Software, vol. 26, no. 2, pp. 14–19, Mar. 2009, doi: 10.1109/MS.2009.47.
  14. M. Jørgensen,
    “Top-down and Bottom-up Expert Estimation of Software Development Effort,” Information and Software Technology, vol. 46, no. 1, pp. 3–16, Jan. 2004, doi: 10.1016/S0950-5849(03)00093-4.
  15. J. Aranda and S. Easterbrook,
    “Anchoring and Adjustment in Software Estimation,” in Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE ’05), ACM, Sept. 2005, pp. 346–355. doi: 10.1145/1081706.1081761.
  16. T. Halkjelsvik and M. Jørgensen,
    “From Origami to Software Development: A Review of Studies on Judgment-Based Predictions of Performance Time,” Psychological Bulletin, vol. 138, no. 2, pp. 238–271, 2012, doi: 10.1037/a0025996.
  17. J. W. Grenning,
    “Planning Poker or How to Avoid Analysis Paralysis while Release Planning.” Renaissance Software Consulting, Apr. 2002. Accessed: Oct. 11, 2026. [Online]. Available: https://wingman-sw.com/papers/PlanningPoker-v1.1.pdf
  18. K. S. Rubin,
    Essential Scrum: A Practical Guide to the Most Popular Agile Process. in Addison-Wesley Signature Series (Cohn). Addison-Wesley Professional, 2012. [Online]. Available: https://innolution.com/essential-scrum
  19. E. Miranda, P. Bourque, and A. Abran,
    “Sizing User Stories Using Paired Comparisons,” Information and Software Technology, vol. 51, no. 9, pp. 1327–1337, Sept. 2009, doi: 10.1016/j.infsof.2009.04.003.
  20. V. Mahnič and T. Hovelja,
    “On Using Planning Poker for Estimating User Stories,” Journal of Systems and Software, vol. 85, no. 9, pp. 2086–2095, Sept. 2012, doi: 10.1016/j.jss.2012.04.005.
  21. N. C. Haugen,
    “An Empirical Study of Using Planning Poker for User Story Estimation,” in Proceedings of AGILE 2006 Conference (AGILE’06), IEEE Computer Society, 2006, pp. 23–34. doi: 10.1109/AGILE.2006.16.
  22. E. Miranda,
    “Improving subjective estimates using paired comparisons,” IEEE Software, vol. 18, no. 1, pp. 87–91, Jan. 2001, doi: 10.1109/52.903173.
  23. R. S. Pressman,
    Software Engineering: A Practitioner’s Approach, 7th ed. McGraw-Hill, 2010.
  24. M. Jørgensen,
    “A Review of Studies on Expert Estimation of Software Development Effort,” Journal of Systems and Software, vol. 70, no. 1–2, pp. 37–60, Feb. 2004, doi: 10.1016/S0164-1212(02)00156-5.
  25. C. Jones,
    “By Popular Demand: Software Estimating Rules of Thumb,” Computer, vol. 29, no. 3, pp. 116–118, Mar. 1996, doi: 10.1109/MC.1996.485905.
  26. K. Moløkken-Østvold and M. Jørgensen,
    “Group Processes in Software Effort Estimation,” Empirical Software Engineering, vol. 9, no. 4, pp. 315–334, Dec. 2004, doi: 10.1023/B:EMSE.0000039882.39206.5a.
  27. D. Root,
    Managing Software Development. Carnegie Mellon University / Innopolis University lecture series, 2014.

Disclaimer: AI is used for text summarization, polishing and explaining. Authors have verified all facts and claims. In case of an error, feel free to file an issue.

This site uses Just the Docs, a documentation theme for Jekyll.