In this study, we report empirical results from forecasting field defect rates and predicting the
number of field defects for a large commercial software system. We find that we are not able to
accurately forecast field defect rates using a combined time-based and metrics-based approach, as
judged by the Theil forecasting statistic. We suggest possible conditions that may have
contributed to the poor results. Next, we use metrics-based approaches to predict the number of
field defects within the six months after deployment. We find that the simple ratios method
produce more accurate predictions than more complex metrics-based methods. Our results are
steps toward quantitatively managing the risks associated with software field defects.
Preferred citation: Paul Luo Lo, Mary Shaw, Jim Herbsleb, P. Santhanam, Bonnie Ray. An Empirical Comparison of Field Defect Modeling Methods, Carnegie Mellon University technical report ISRI-06-102, May 2005