A Critique of How We Measure and Interpret the Accuracy of Software Development Effort Estimation
In: 1st International Workshop on Software Productivity Analysis and Cost Estimation, ed. by Jacky Keung, pp. 15-22, Information Processing Society of Japan (ISBN: ISBN 978-4-915256-72-1 C3040)
This paper criticizes current practice regarding the measurement and interpretation of the accuracy of software development effort estimation. The shortcomings we discuss are related to: 1) the meaning of ‘effort estimate’, 2) the meaning of ‘estimation accuracy’, 3) estimation of moving targets, and 4) assessment of the estimation process, and not only the discrepancy between the estimated and the actual effort, to evaluate estimation skill. It is possible to correct several of the discussed shortcomings by better practice. However, there are also inherent problems related to both laboratory and field analyses of the accuracy of software development effort estimation. It is essential that both software researchers and professionals are aware of these problems and their implications for the analysis of the measurement of effort estimation accuracy.