Validation, Traceability and Change

Once requirements exist, three activities keep them useful: checking them, linking them to what they produced, and deciding what to do when they change. The third is the one most often got backwards.

1. Checking a requirement set

Validation asks whether these are the right requirements; verification asks whether the system meets them — the distinction is on requirements engineering. What a review looks for is narrower than “read it carefully”: ambiguity; inconsistency, omission, including anything still marked to be determined, since a specification containing one is not complete [1]; a named source who confirms the written form is what they meant; and a way to test it. If nothing can determine whether the software meets a requirement, remove or revise it.

A V, with validation on the way down and verification on the way up Stakeholders, then the build. [2]

2. Traceability

Traceability tracks a requirement in two directions: backward to the stakeholder, document or regulation that motivated it, and forward to the design, code and tests that satisfy it. For example, “R-005: user data shall be encrypted at rest” traces back to a data-protection obligation, and forward to an encryption component and an AES-256 storage test. The backward link lets you re-decide the requirement when the obligation changes; the forward link tells you what breaks.

3. Change is what successful products do

Commercial word processors and spreadsheets grew from under 300 function points to over 5,000 in ten years [3] — not a failure of discipline, but what a surviving product looks like.

Maintenance effort: fixing defects against adding what users now want Mostly enhancement, not repair. [4]

And most change is not repair. Corrective maintenance is only 10–15% of maintenance work; functional enhancements are over 60% of changes, and 40% of those enhancements come from learning [4] — figures that are Edberg and Olfman’s, relayed by Kelly. Specifying a system also changes the system people want: interviewing people about their work “will cause people to reflect on what they are doing… and whether things can be done better.”

4. Measure change before arguing about it

Volatility is “the ratio of requirements change… to the total number of requirements for a given period of time” — a number any project can produce from its change log [5].

The cost of change rises, and with it the bar a change must clear The price rises; the door stays open. [4]

Recording what changed is easy. Recording why and where it came from turns a change log into an analysis: changes originating in design reviews mean the reviews are working; changes originating in support calls mean the elicitation did not. In the studied release, volatility peaked at 16.85% as requirements analysis completed and the reviews ran — which is discovery, not instability. A project that schedules no review has the same changes, found later and priced higher.

5. What a late change actually costs

Three independent methods on real aerospace programmes agree that the cost to fix an error grows exponentially with the phase in which it is found [6]. One aircraft programme’s own accounting, over 231 true errors, gives it in money: an error caught in definition cost $22,632; the same class of error found in operations cost $3,558,215.

Cost to fix one requirements error by the phase it was found in, rising in a straight line on a log axis A straight line on a log axis is exponential. [6]

There is no single multiplier. The familiar figure carries its own hedge — a problem found after delivery is “often 100 times more expensive” — and the same authors put small non-critical systems “more like 5:1 than 100:1” [7]. Measured escalation to operations runs 29× on real spacecraft and 157–186× on real aircraft.

The mechanism is the useful part: projects spend about 40–50% of their effort on avoidable rework, one named source of which is hastily specified requirements.

6. The decision rule the cost curve licenses

Not refusal. As Kelly puts it: “changes that come along later are more disruptive but this doesn’t imply they are valueless, only that they must be worth more if they are to be worthwhile implementing.” The rule is a rising value threshold, not a closing door — read as a licence to say no, it makes you “the people who always say ‘No’“.

Two mechanisms implement it: a written impact statement listing alternatives with their costs, so the decision rests with the requestor, not the manager [8]; and a deferral device, where ideas go into a bank and are decided at the next checkpoint rather than inside a cycle.

How solid is this?

  • Where it comes from. One triangulated aerospace cost study, one single-release volatility case, one synthesis column, one opinion essay whose percentages belong to a study it cites.
  • What is contested. The 100× figure: published studies disagree by a factor of nearly seven at the test phase alone, and Boehm’s own 1981 operations figure is a range of 40–1000×.
  • What we do not hold. The volatility percentages are one waterfall release at one company and are not a benchmark.

Acknowledgments

This page adapts material from lectures by Eduardo Miranda and David Root [9] on software project management.

References

  1. ISO/IEC/IEEE,
    ISO/IEC/IEEE 29148:2018 — Systems and Software Engineering — Life Cycle Processes — Requirements Engineering. International Organization for Standardization, 2018. [Online]. Available: https://www.iso.org/standard/72089.html
  2. B. Nuseibeh and S. Easterbrook,
    “Requirements Engineering: A Roadmap,” in Proceedings of the Conference on The Future of Software Engineering (ICSE ’00), ACM, May 2000, pp. 35–46. doi: 10.1145/336512.336523.
  3. C. Jones,
    “Strategies for Managing Requirements Creep,” Computer, vol. 29, no. 6, pp. 92–94, June 1996, doi: 10.1109/2.507640.
  4. A. Kelly,
    “Why Do Requirements Change?,” Overload, vol. 12, no. 59, Feb. 2004, [Online]. Available: https://accu.org/journals/overload/12/59/kelly_319/
  5. N. Nurmuliani, D. Zowghi, and S. Fowell,
    “Analysis of Requirements Volatility During Software Development Life Cycle,” in Proceedings of the 2004 Australian Software Engineering Conference (ASWEC 2004), IEEE, 2004, pp. 28–37. doi: 10.1109/ASWEC.2004.1290455.
  6. B. Haskins, J. Stecklein, B. Dick, G. Moroney, R. Lovell, and J. Dabney,
    “Error Cost Escalation Through the Project Life Cycle,” INCOSE International Symposium, vol. 14, no. 1, pp. 1723–1737, June 2004, doi: 10.1002/j.2334-5837.2004.tb00608.x.
  7. B. Boehm and V. R. Basili,
    “Software Defect Reduction Top 10 List,” Computer, vol. 34, no. 1, pp. 135–137, Jan. 2001, doi: 10.1109/2.962984.
  8. R. K. Wysocki and R. McGary,
    Effective Project Management: Traditional, Adaptive, Extreme, 3rd ed. Wiley Publishing, 2003.
  9. D. Root,
    Managing Software Development. Carnegie Mellon University / Innopolis University lecture series, 2014.

Disclaimer: AI is used for text summarization, polishing and explaining. Authors have verified all facts and claims. In case of an error, feel free to file an issue.

This site uses Just the Docs, a documentation theme for Jekyll.