Skip to main content
SpringerLink
Log in

Empirical assessment of two approaches for specifying software product line use case scenarios

  • Theme Section Paper
  • Published:
Software & Systems Modeling Aims and scope Submit manuscript

Abstract

Modularity benefits, including the independent maintenance and comprehension of individual modules, have been widely advocated. However, empirical assessments to investigate those benefits have mostly focused on source code, and thus, the relevance of modularity to earlier artifacts is still not so clear (such as requirements and design models). In this paper, we use a multimethod technique, including designed experiments, to empirically evaluate the benefits of modularity in the context of two approaches for specifying product line use case scenarios: PLUSS and MSVCM. The first uses an annotative approach for specifying variability, whereas the second relies on aspect-oriented constructs for separating common and variant scenario specifications. After evaluating these approaches through the specifications of several systems, we find out that MSVCM reduces feature scattering and improves scenario cohesion. These results suggest that evolving a product line specification using MSVCM requires only localized changes. On the other hand, the results of six experiments reveal that MSVCM requires more time to derive the product line specifications and, contrasting with the modularity results, reduces the time to evolve a product line specification only when the subjects have been well trained and are used to the task of evolving product line specifications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

Notes

  1. This panel occurred at the 10th Software Product Line Conference.

  2. http://bit.ly/cQ7ZmU.

  3. Actually, the effort was collected in milliseconds. However, to facilitate the reader to comprehend our results, we discuss our findings in this paper either in seconds or in minutes.

References

  1. Alférez, M., Bonifácio, R., Teixeira, L., Accioly, P., Kulesza, U., Moreira, A., Araújo, J.a., Borba, P.: Evaluating scenario-based SPL requirements approaches: the case for modularity, stability and expressiveness. Requir. Eng. pp. 1–22 (2013). doi:10.1007/s00766-013-0184-5

  2. Alferez, M., et al.: A model-driven approach for software product lines requirements engineering. In: Proceedings of the Twentieth International Conference on Software Engineering & Knowledge Engineering (SEKE’2008), pp. 779–784. San Francisco, USA (2008)

  3. Alférez, M., et al.: Multi-view composition language for software product line requirements. In: 2nd International Conference on Software Language Engineering. Denver, USA (2009)

  4. Apel, S., Leich, T., Saake, G.: Aspectual mixin layers: aspects and features in concert. In: 28th International conference on Software engineering, pp. 122–131. ACM, New York, NY, USA (2006). doi:10.1145/1134285.1134304

  5. Basili, V.: The role of experimentation in software engineering: past, current, and future. In: Software Engineering, 1996, Proceedings of the 18th International Conference on, pp. 442–449 (1996). doi:10.1109/ICSE.1996.493439

  6. Basili, V., Shull, F., Lanubile, F.: Building knowledge through families of experiments. IEEE Trans. Softw. Eng. 25(4), 456–473 (1999). doi:10.1109/32.799939

    Article  Google Scholar 

  7. Bertolino, A., Gnesi, S.: Use case-based testing of product lines. In: 9th European Software Engineering Conference (ESEC/FSE), pp. 355–358. ACM, New York (2003). doi:10.1145/940071.940120

  8. Bonifácio, R., Borba, P.: Modeling scenario variability as crosscutting mechanisms. In: AOSD ’09: Proceedings of the 8th ACM International Conference on Aspect-Oriented Software Development, pp. 125–136. ACM, New York, NY (2009). doi:10.1145/1509239.1509258

  9. Bonifácio, R., Borba, P., Soares, S.: On the benefits of variability management as crosscutting. In: Early Aspects Workshop at AOSD. Brussels, Belgium (2008)

  10. Borba, P., Teixeira, L., Gheyi, R.: A theory of software product line refinement. Theoretical Computer Science 455(0), 2–30 (2012). doi:10.1016/j.tcs.2012.01.031. ¡ce:title? International Colloquium on Theoretical Aspects of Computing¡ce:title¿

  11. Borenstein, M., Hedges, L., Higgins, J., Rothstein, H.: Introduction to Meta-Analysis. Statistics in Practice. Wiley, New York (2009)

    Book  MATH  Google Scholar 

  12. Box, G., Hunter, J., Hunter, W.: Statistics for Experimenters: Design, Innovation, and Discovery, 2nd edn. Wiley-Interscience, New York (2004)

    MATH  Google Scholar 

  13. Chaudhuri, S.: Microsoft conference management system. on-line (2009). http://cmt.research.microsoft.com/cmt/

  14. Chitchyan, R., Greenwood, P., Sampaio, A., Rashid, A., Garcia, A.F., da Silva, L.F.: Semantic vs. syntactic compositions in aspect-oriented requirements engineering: an empirical study. In: K.J. Sullivan (ed.) Proceedings of the 8th International Conference on Aspect-Oriented Software Development, AOSD 2009, Charlottesville, Virginia, USA, March 2–6, 2009, pp. 149–160. ACM (2009). doi:10.1145/1509239.1509260

  15. Ciolkowski, M.: What do we know about perspective-based reading? an approach for quantitative aggregation in software engineering. In: Empirical Software Engineering and Measurement, 2009. ESEM 2009. 3rd International Symposium on, pp. 133–144 (2009)

  16. Clements, P., Northrop, L.: Software Product Lines: Practices and Patterns. Addison-Wesley Professional, Reading (2001)

    Google Scholar 

  17. Czarnecki, K., Antkiewicz, M.: Mapping features to models: A template approach based on superimposed variants. In: Generative Programming and Component Engineering (GPCE), LNCS, vol. 3676, pp. 422–437. Springer (2005). doi:10.1007/11561347_28

  18. Czarnecki, K., Eisenecker, U.: Generative Programming: Methods, Tools, and Applications. ACM Press/Addison-Wesley Publishing Co., New York (2000)

    Google Scholar 

  19. Czarnecki, K., Pietroszek, K.: Verifying feature-based model templates against well-formedness ocl constraints. In: 5th International Conference on Generative Programming and Component Engineering, pp. 211–220. ACM, New York, NY (2006). doi:10.1145/1173706.1173738

  20. Deligiannis, I., Shepperd, M., Roumeliotis, M., Stamelos, I.: An empirical investigation of an object-oriented design heuristic for maintainability. J. Syst. Softw. 65(2), 127–139 (2003). doi:10.1016/S0164-1212(02)00054-7

    Article  Google Scholar 

  21. Deligiannis, I.S., et al.: A review of experimental investigations into object-oriented technology. Empir. Softw. Eng. 7, 193–231 (2002)

    Article  MATH  Google Scholar 

  22. Do, H., Mirarab, S., Tahvildari, L., Rothermel, G.: The effects of time constraints on test case prioritization: A series of controlled experiments. IEEE Trans. Softw. Eng. 36(5), 593–617 (2010). doi:10.1109/TSE.2010.58

    Article  Google Scholar 

  23. Eaddy, M.: An empirical assessment of the crosscutting concern problem. Ph.D. thesis, Graduate School of Arts and Sciences, Columbia University (2008)

  24. Eaddy, M., Aho, A., Murphy, G.: Identifying, assigning, and quantifying crosscutting concerns. In: First Workshop on Assessment of Contemporary Modularization Techniques (ACOM). Minneapolis, USA (2007)

  25. Eaddy, M., Zimmermann, T., Sherwood, K.D., Garg, V., Murphy, G.C., Nagappan, N., Aho, A.V.: Do crosscutting concerns cause defects? IEEE Trans. Softw. Eng. 34(4), 497–515 (2008). doi:10.1109/TSE.2008.36

    Article  Google Scholar 

  26. Eriksson, M., Borstler, J., Borg, K.: The pluss approach - domain modeling with features, use cases and use case realizations. In: 9th International Conference on Software Product Lines, pp. 33–44. LNCS (2005)

  27. Eriksson, M., Borstler, J., Borg, K.: Software product line modeling made practical. Commun. ACM 49(12), 49–54 (2006). doi:10.1145/1183236.1183265

    Article  Google Scholar 

  28. Eriksson, M., Börstler, J., Borg, K.: Managing requirements specifications for product lines: An approach and industry case study. J. Syst. Softw. 82, 435–447 (2009)

    Article  Google Scholar 

  29. Everitt, B., Hothorn, T.: A Handbook of Statistical Analyses Using R. Chapman & Hall/CRC (2006). http://CRAN.R-project.org/package=HSAUR

  30. Figueiredo, E., et al.: Evolving software product lines with aspects: an empirical study on design stability. In: 30th International Conference on Software Engineering, pp. 261–270. ACM, New York, NY (2008). doi:10.1145/1368088.1368124

  31. Figueiredo, E., et al.: On the Maintainability of Aspect-Oriented Software: A Concern-Oriented Measurement Framework. Software Maintenance and Reengineering, 2008. CSMR 2008. 12th European Conference on pp. 183–192 (2008)

  32. Filho, F.C., Cacho, N., Figueiredo, E., ao, R.M., Garcia, A., Rubira, C.M.F.: Exceptions and aspects: the devil is in the details. In: 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 152–162 (2006)

  33. Garcia, A., et al.: Modularizing design patterns with aspects: a quantitative study. 4th International Conference on Aspect-oriented Software Development 14(18), 3–14 (2005)

  34. Gheyi, R., Massoni, T., Borba, P.: A theory for feature models in alloy. In: First Alloy Workshop, pp. 71–80. Portland, United States (2006)

  35. Glass, R.L.: The software-research crisis. Softw. IEEE 11(6), 42–47 (1994). doi:10.1109/52.329400

    Article  Google Scholar 

  36. Greenwood, P., et al.: On the impact of aspectual decompositions on design stability: An empirical study. 21st European Conference on Object-Oriented Programming (2007)

  37. Group, Z.: Open conference management system. on-line (2009). http://www.openconf.com/

  38. Hanenberg, S., Kleinschmager, S., Josupeit-Walter, M.: Does aspect-oriented programming increase the development speed for crosscutting code? An empirical study. In: ESEM ’09: Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement, pp. 156–167. IEEE Computer Society, Washington, DC, USA (2009). doi:10.1109/ESEM.2009.5316028

  39. Juristo, N., Moreno, A.: Basics of Software Engineering Experimentation. Kluwer Academic Pub, Boston (2001)

    Book  MATH  Google Scholar 

  40. Kang, K.C., Cohen, S.G., Hess, J.A., Novak, W.E., Peterson, A.S.: Feature-oriented domain analysis (foda) feasibility study. Tech. Rep. CMU/SEI-90-TR-21, Software Engineering Institute (1990)

  41. Kästner, C., Apel, S., Kuhlemann, M.: Granularity in software product lines. In: 30th International Conference on Software Engineering, pp. 311–320. ACM, New York (2008). doi:10.1145/1368088.1368131

  42. Kiczales, G., Hilsdale, E., Hugunin, J., Kersten, M., Palm, J., Griswold, W.: Getting started with aspect. Commun. ACM 44(10), 59–65 (2001). doi:10.1145/383845.383858

    Article  Google Scholar 

  43. Kiczales, G., Hilsdale, E., Hugunin, J., Kersten, M., Palm, J., Griswold, W.G.: An overview of AspectJ. In: ECOOP ’01: Proceedings of the 15th European Conference on Object-Oriented Programming, pp. 327–353. Springer, London (2001)

  44. Kienzle, J., et al.: Crisis management systems (2009). http://www.cs.mcgill.ca/joerg/taosd/TAOSD/

  45. Kitchenham, B., Pfleeger, S., Pickard, L., Jones, P., Hoaglin, D., El Emam, K., Rosenberg, J.: Preliminary guidelines for empirical research in software engineering. IEEE Trans. Softw. Eng. 28(8), 721–734 (2002). doi:10.1109/TSE.2002.1027796

    Article  Google Scholar 

  46. Kitchenham, B., Pfleeger, S.L., Fenton, N.: Towards a framework for software measurement validation. IEEE Trans. Softw. Eng. 21(12), 929–944 (1995). doi:10.1109/32.489070

    Article  Google Scholar 

  47. Konstantopoulos, S., Hedges, L.V.: Analyzing effect sizes: Fixed-effects models. The handbook of research synthesis and meta-analysis pp. 279–293 (2009)

  48. Krueger, C.W.: Easing the transition to software mass customization. In: Revised Papers from the 4th International Workshop on Software Product-Family Engineering, PFE ’01, pp. 282–293. Springer, London (2002). http://dl.acm.org/citation.cfm?id=648114.748909

  49. Krueger, C.W.: New methods in software product line practice. Commun. ACM 49(12), 37–40 (2006). doi:10.1145/1183236.1183262

    Article  Google Scholar 

  50. Kuehl, R., Kuehl, R.: Design of experiments: statistical principles of research design and analysis. Duxbury-Thomson Learning (2000)

  51. Lagaisse, B., Win, B.D., Joosen, W., Oeyen, J.V.: E-finance case study: analysis and requirements. Tech. Rep. CW438, Department of Computer Science, Katholieke Universiteit Leuven (2006). http://www.cs.kuleuven.be/publicaties/rapporten/cw/CW438.abs.html

  52. Lemos, O., Ferrari, F., Silveira, F., Garcia, A.: Development of auxiliary functions: Should you be agile? an empirical assessment of pair programming and test-first programming. In: Software Engineering (ICSE), 2012 34th International Conference on, pp. 529–539 (2012). doi:10.1109/ICSE.2012.6227163

  53. Lima, L., Iyoda, J., Sampaio, A., Aranha, E.: Test case prioritization based on data reuse an experimental study. In: ESEM ’09: Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement, pp. 279–290. IEEE Computer Society, Washington, DC (2009). doi:10.1109/ESEM.2009.5315980

  54. Linden, F.J.v.d., Schmid, K., Rommes, E.: Software Product Lines in Action: The Best Industrial Practice in Product Line Engineering. Springer, New York (2007)

  55. Masuhara, H., Kiczales, G.: Modeling crosscutting in aspect-oriented mechanisms. In: European Conference on Object-Oriented Programming (ECOOP), Lecture Notes in Computer Science, pp. 2–28. Springer (2003)

  56. Pohl, K., Böckle, G., van der Linden, F.J.: Software Product Line Engineering: Foundations, Principles and Techniques. Springer, New York (2005)

    Book  MATH  Google Scholar 

  57. Pohl, K., Metzger, A.: The eshop product line. Online: http://www.sei.cmu.edu/splc2006/eShop.pdf

  58. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2011). http://www.R-project.org/. ISBN 3-900051-07-0

  59. Ramsey, F., Schafer, D.: The Statistical Sleuth: A Course in Methods of Data Analysis. Duxbury Press Belmont, California (1997)

    MATH  Google Scholar 

  60. Sampaio, A., Greenwood, P., Garcia, A., Rashid, A.: A comparative study of aspect-oriented requirements engineering approaches. In: Empirical Software Engineering and Measurement, 2007. ESEM 2007. First International Symposium on, pp. 166–175 (2007). doi:10.1109/ESEM.2007.15

  61. Sjoberg, D.I., Yamashita, A., Anda, B., Mockus, A., Dyba, T.: Quantifying the effect of code smells on maintenance effort. IEEE Transactions on Software Engineering 99(PrePrints), 1 (2012). doi:10.1109/TSE.2012.89

  62. Soares, S., Borba, P., Laureano, E.: Distribution and persistence as aspects. Softw Pract Exp 36(7), 711–759 (2006)

    Article  Google Scholar 

  63. Soares, S., Laureano, E., Borba, P.: Implementing distribution and persistence aspects with aspectj. SIGPLAN Not. 37(11), 174–190 (2002). doi:10.1145/583854.582437

    Article  Google Scholar 

  64. Whittle, J., Araujo, J.: Scenario modelling with aspects. IEE Proc Softw 151(4), 157–171 (2004). doi:10.1049/ip-sen:20040921

    Article  Google Scholar 

  65. Whittle, J., Moreira, A., ao Araújo, J., Jayaraman, P., Elkhodary, A., Rabbi, R.: An expressive aspect composition language for uml state diagrams. In: International Conference on Model Driven Engineering, Languages and Systems (MODELS’2007), LNCS, vol. 4735, pp. 514–528. Springer (2007)

  66. Wohlin, C., Runeson, P., Host, M., Ohlsson, C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering: An Introduction. Kluver, Norwell (2000)

    Book  MATH  Google Scholar 

  67. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer Science & Business Media, New York (2012)

    Book  MATH  Google Scholar 

Download references

Acknowledgments

This work was partially supported by the National Institute of Science and Technology for Software Engineering [INES (http://www.ines.org.br], funded by CNPq and FACEPE, Grants 573964/2008-4 and APQ-1037-1.03/08. The first author is supported by the Grants CNPq CT-INFO 17/2007.

Author information

Authors and Affiliations

  1. Computer Science Department, University of Brasília, Brasília, Brazil

    Rodrigo Bonifácio

  2. Informatics Center, Federal University of Pernambuco, Recife, Brazil

    Paulo Borba & Paola Accioly

  3. Statistics Department, Federal University of Pernambuco, Recife, Brazil

    Cristiano Ferraz

Authors
  1. Rodrigo Bonifácio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paulo Borba
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cristiano Ferraz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paola Accioly
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rodrigo Bonifácio.

Additional information

Communicated by Andrzej Wąsowski and Thorsten Weyer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bonifácio, R., Borba, P., Ferraz, C. et al. Empirical assessment of two approaches for specifying software product line use case scenarios. Softw Syst Model 16, 97–123 (2017). https://doi.org/10.1007/s10270-015-0471-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-015-0471-3

Keywords

Search

Navigation