Skip to main content
SpringerLink
Log in

A Comparative Study of Cost Estimation Models for Web Hypermedia Applications

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Software cost models and effort estimates help project managers allocate resources, control costs and schedule and improve current practices, leading to projects finished on time and within budget. In the context of Web development, these issues are also crucial, and very challenging given that Web projects have short schedules and very fluidic scope. In the context of Web engineering, few studies have compared the accuracy of different types of cost estimation techniques with emphasis placed on linear and stepwise regressions, and case-based reasoning (CBR). To date only one type of CBR technique has been employed in Web engineering. We believe results obtained from that study may have been biased, given that other CBR techniques can also be used for effort prediction. Consequently, the first objective of this study is to compare the prediction accuracy of three CBR techniques to estimate the effort to develop Web hypermedia applications and to choose the one with the best estimates. The second objective is to compare the prediction accuracy of the best CBR technique against two commonly used prediction models, namely stepwise regression and regression trees. One dataset was used in the estimation process and the results showed that the best predictions were obtained for stepwise regression.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ambler, S. W. 2002. Lessons in agility from Internet-based development. IEEE Software (Mar.-Apr.): 66-73.

  • Angelis, L., and Stamelos, I. 2000. A simulation tool for efficient analogy based cost estimation. Empirical Software Engineering 5: 35-68.

    Google Scholar 

  • Balasubramanian, V., Isakowitz, T., and Stohr, E. A. 1995. RMM: A methodology for structured hypermedia design. CACM 38(Aug.): 8.

    Google Scholar 

  • Boehm, B. 1981. Software Engineering Economics. Englewood Cliffs, N.J.: Prentice-Hall.

    Google Scholar 

  • Botafogo, R., Rivlin, A. E., and Shneiderman, B. 1992. Structural analysis of hypertexts: Identifying hierarchies and useful metrics. ACM TOIS 10(2): 143-179.

    Google Scholar 

  • Briand, L. C., El-Emam, K., Surmann, D., Wieczorek, I., and Maxwell, K. D. 1999. An assessment and comparison of common cost estimation modeling techniques. Proc. ICSE 1999. Los Angeles, USA, 313-322.

  • Briand, L. C., Langley, T., and Wieczorek, I. 2000. A replicated assessment and comparison of common software cost modeling techniques. Proc. ICSE 2000. Limerick, Ireland, pp. 377-386.

  • Brieman, L., Friedman, J., Olshen, R., and Stone, C. 1984. Classification and Regression Trees. Belmont: Wadsworth Inc.

    Google Scholar 

  • Christodoulou, S. P., Zafiris, P. A., and Papatheodorou, T. S. 2000. Proc. 2nd ICSE Workshop Web Eng, 75-92.

  • Coda, F., Ghezzi, C., Vigna, G., and Garzotto, F. 1998. Towards a software engineering approach to web site development. Proc. 9th International Workshop on Software Specification and Design, 8-17.

  • Conte, S., Dunsmore, H., and Shen, V. 1986. Software engineering metrics and models. Menlo Park, CA: Benjamin/Cummings.

    Google Scholar 

  • COSMIC, 1999. COSMIC-FFP Measurement manual, version 2.0, http://www.cosmicon.com.

  • Cowderoy, A. J. C., Donaldson, A. J. M., and Jenkins, J. O. 1998. A metrics framework for multimedia creation. Proc. 5th IEEE International Software Metrics Symposium. Maryland, USA.

  • Cowderoy, A. J. C. 2000. Measures of size and complexity for web-site content. Proc. Combined 11th ESCOM Conference and the 3rd SCOPE conference on Software Product Quality. Munich, Germany, 423-431.

  • DeMarco, T. 1982. Controlling Software Projects: Management, Measurement and Estimation. New York: Yourdon.

    Google Scholar 

  • Dolado, J. J. 2001. On the problem of the software cost function. IST 43: 61-72.

    Google Scholar 

  • Fielding, R. T., and Taylor, R. N. 2000. Principled design of the modern Webarchitectu re. Proc. ICSE. ACM. New York, NY, USA, 407-416.

  • Finnie, G. R., Wittig, G. E., and Desharnais, J.-M. 1997. A comparison of software effort estimation techniques: Using function points with neural networks, case-based reasoning and regression models. Journal of Systems and Software 39: 281-289.

    Google Scholar 

  • Garzotto, F., Paolini, P., and Schwabe, D. 1993. HMD-A model-based approach to hypertext application design. ACM TOIS 11(January): 1.

    Google Scholar 

  • Gray, A., and MacDonell, S. 1997a. Applications of Fuzzy logic to software metric models for development effort estimation. Proc. Annual Meetingof the North American Fuzzy Information Processing Society-NAFIPS. Syracuse NY, USA, IEEE, 394-399.

    Google Scholar 

  • Gray, A. R., and MacDonell, S. G. 1997b. A comparison of model building techniques to develop predictive equations for software metrics. Information and Software Technology 39: 425-437.

    Google Scholar 

  • Gray, R., MacDonell, S. G., and Shepperd, M. J. 1999. Factors systematically associated with errors in subjective estimates of software development effort: The stability of expert judgment. Proc. IEEE 6th Metrics Symposium.

  • Hughes, R. T. 1997. An empirical investigation into the estimation of software development effort, PhD thesis, Dept. of Computing, the University of Brighton, UK.

    Google Scholar 

  • Humphrey, W. S. 1995. A Discipline for Software Engineering. SEI Series in Software Engineering, Addison-Wesley.

  • Jeffery, R., Ruhe, M., and Wieczorek, I. 2000. A comparative study of two software development cost modeling techniques using multi-organizational and company-specific data. Information and Software Technology 42: 1009-1016.

    Google Scholar 

  • Jeffery, R., Ruhe, M., and Wieczorek, I. 2001. Using public domain metrics to estimate software development effort. Proc. IEEE 7th Metrics Symposium. London, UK, 16-27.

  • Johnson, P. M., and Disney, A. M. 1999. A critical analysis of PSP data quality: Results from a case study. Journal of Empirical Software Engineering (Dec.).

  • Kadoda, G., Cartwright, M., and Shepperd, M. J. 2001. Issues on the effective use of CBR technology for software project prediction. Proc. 4th international conference on case-based reasoning. Vancouver, Canada, July/August, 276-290.

  • Kadoda, G., Cartwright, M., Chen, L., and Shepperd, M. J. 2000. Experiences using case-based reasoning to predict software project effort. Proc. EASE 2000 Conference. Keele, UK.

  • Kemerer, C. F. 1987. An empirical validation of software cost estimation models. CACM 30(5): 416-429.

    Google Scholar 

  • Kinnear, P. R., and Gray, C. D. 1999. SPSS for Windows Made Simple. 3rd edition, Psychology Press Ltd.

  • Kirsopp, C., and Shepperd, M. 2001. Making Inferences with Small Numbers of Training Sets, January, TR02-01., Bournemouth University.

  • Kitchenham, B. A., Pickard, L. M., MacDonell, S. G., and Shepperd, M. J. 2001. What accuracy statistics really measure. IEE Proc.-Software Engineering. 148(June): 3.

    Google Scholar 

  • Kitchenham, B. A., Pickard, L., and Pfleeger, S. L. 1995. Case studies for method and tool evaluation. IEEE Software (July): 52-62.

  • Kok, P., Kitchenham, B. A., and Kirakowski, J. 1990. The MERMAID approach to software cost estimation, ESPRIT Annual Conference, Brussels, 296-314.

  • Kumar, S., Krishna, B. A., and Satsangi, P. S. 1994. Fuzzy systems and neural networks in software engineering project management. Journal of Applied Intelligence 4: 31-52.

    Google Scholar 

  • Maxwell, K. D. 2001. Collecting data for comparability: Benchmarking software development productivity. IEEE Software (Sept.-Oct.): 22-25.

  • Mendes, E., Counsell, S., and Mosley, N. 2000. Measurement and effort prediction of web applications. Proc. 2nd ICSE Workshop on Web Engineering (June). Limerick, Ireland.

  • Mendes, E., Counsell, S., and Mosley, N. 2001a. Towards the prediction of development effort for hypermedia applications. Proc. ACM Hypertext'01 Conference. Aahrus, Denmark, ACM.

  • Mendes, E., Mosley, N., and Counsell, S. 2001b. Web metrics-estimating design and authoring effort. IEEE Multimedia. Special Issue on Web Engineering, (Jan.-Mar.): 50-57.

  • Mendes, E., Mosley, N., and Counsell, S. 2002a. A comparison of size measures for predicting web design and authoring effort. IEE Proc.-Software Engineering 149(3): 77-85.

    Google Scholar 

  • Mendes, E., Mosley, N., and Watson, I. 2002b. A comparison of case-based reasoning approaches to web hypermedia project cost estimation. Proc. 11th International World-Wide Web Conference, Hawaii.

  • Michau, F., Gentil, S., and Barrault, M. 2001. Expected benefits of web-based learning for engineering education: Examples in control engineering. European Journal of Engineering Education 26(2): 151-168.

    Google Scholar 

  • Myrtveit, I., and Stensrud, E. 1999. A controlled experiment to assess the benefits of estimating with analogy and regression models. IEEE Trans. on Software Engineering 25(4): 510-525.

    Google Scholar 

  • Okamoto, S., and Satoh, K. 1995. An average-case analysis of k-nearest neighbor classifier. In CBR Research and Development, Veloso, M., & Aamodt, A. (eds.) Lecture Notes in Artificial Intelligence 1010, Springer-Verlag.

  • Pickard, L. M., Kitchenham, B. A., and Linkman, S. J. 1999. An investigation of analysis techniques for software datasets. Proc. 6th International Symposium on Software Metrics. Los Alamitos, CA: IEEE Computer Society Press.

    Google Scholar 

  • Pressman, R. S. 2000. What a tangled web we weave. IEEE Software (Jan.-Feb.): 18-21.

  • Putnam, L. H. 1978. A general empirical solution to the macro sizing and estimating problem. IEEE Trans. on Software Engineering SE-4(4): 345-361.

    Google Scholar 

  • Ranwez, S., Leidig, T., and Crampes, M. 2000. Formalization to improve lifelong learning. Journal of Interactive Learning Research 11(3-4): 389-409. Assoc. Advancement Comput. Educ., USA.

    Google Scholar 

  • Reifer, D. J. 2000. Web development: Estimating quick-to-market software. IEEE Software (Nov.-Dec.): 57-64.

  • Reifer, D. J. 2002. Ten deadly risks in internet and intranet software development. IEEE Software (Mar.-Apr.): 12-14.

  • Schofield, C. 1998. An empirical investigation into software estimation by analogy, PhD thesis, Department of Computing, Bournemouth University, UK.

    Google Scholar 

  • Schroeder, L., Sjoquist, D., and Stephan, P. 1986. Understanding Regression Analysis: An Introductory Guide. No. 57. In Series: Quantitative Applications in the Social Sciences, CA, USA: Sage Publications, Newbury Park.

    Google Scholar 

  • Schulz, S. 1999. CBR-works-A state-of-the-art shell for case-based application building. Proc. of the German Workshop on Case-Based Reasoning. Lecture Notes in Artificial Intelligence. Springer-Verlag.

  • Schwabe, D., and Rossi, G. 1994. From domain models to hypermedia applications: An object-oriented approach. Proc. International Workshop on Methodologies for Designing and Developing Hypermedia Applications (Sept.).

  • Selby, R. W., and Porter, A. A. 1998. Learning from examples: generation and evaluation of decision trees for software resource analysis. IEEE Trans. on Software Engineering 14: 1743-1757.

    Google Scholar 

  • Shepperd, M. J., and Cartwright, M. Predicting with sparse data. 2001. Proc. 7th IEEE Software Metrics Symposium, 28-39.

  • Shepperd, M. J., and Schofield, C. 1997. Estimating software project effort using analogies. IEEE Trans. on Software Engineering 23(11): 736-743.

    Google Scholar 

  • Shepperd, M. J., and Kadoda, G. 2001. Using simulation to evaluate prediction techniques. Proc. IEEE 7th International Software Metrics Symposium. London, UK, 349-358.

  • Shepperd, M. J., Schofield, C., and Kitchenham, B. 1996. Effort estimation using analogy. Proc. ICSE-18. Berlin.

  • Spiro, R. J., Feltovich, P. J., Jacobson, M. J., and Coulson, R. L. 1995. Cognitive flexibility, constructivism, and hypertext: Random access instruction for advanced knowledge acquisition in Ill-structured domains. In L. Steffe & J. Gale (eds.), Constructivism, Hillsdale, N.J.: Erlbaum.

    Google Scholar 

  • Srinivasan, K., and Fisher, D. 1995. Machine learning approaches to estimating software development effort. IEEE Trans. on Software Engineering 21: 126-137.

    Google Scholar 

  • Stensrud, E., Foss, T., Kitchenham, B. A., and Myrtveit, I. 2002. An empirical validation of the relationship between the magnitude of relative error and project size. Proc. IEEE 8th Metrics Symposium 3: 12.

    Google Scholar 

  • Tosca, S. P. 1999. The lyrical quality of links hypertext. Proc. 10th ACM Hypertext Conference. ACM. 217-218.

  • Watson, I. 1997. Applying Case-Based Reasoning: Techniques for Enterprise Systems. San Francisco, USA: Morgan Kaufmann.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, The University of Auckland, Auckland, New Zealand

    Emilia Mendes

  2. Computer Science Department, The University of Auckland, Auckland, New Zealand

    Ian Watson

  3. Statistics Department, The University of Auckland, Auckland, New Zealand

    Chris Triggs

  4. MxM Technology, Auckland, New Zealand

    Nile Mosley

  5. Computer Science Department, Birkbeck College, University of London, London, UK

    Steve Counsell

Authors
  1. Emilia Mendes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ian Watson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chris Triggs
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nile Mosley
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Steve Counsell
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mendes, E., Watson, I., Triggs, C. et al. A Comparative Study of Cost Estimation Models for Web Hypermedia Applications. Empirical Software Engineering 8, 163–196 (2003). https://doi.org/10.1023/A:1023062629183

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1023062629183

Search

Navigation