Regression testing: techniques and future research directions

H.B. Moroz, A.B. Plys

Abstract


The most crucial phase in the software life cycle is maintenance phase, in which the development team is supposed to maintain the software which is delivered to the clients by them. Software maintenance results for the reasons like error corrections, enhancement of capabilities, deletion of obsolete capabilities and optimization. Now the changed or modified software needs testing known as regression testing. This paper surveys the two main groups of methods of regression testing (regression test selection methods and methods of prioritization) and discuss open problems and potential directions for future research.

Prombles in programming 2014; 2-3: 133-145


References


Pressman R. Software Engineering: A Practitioner’s Approach. New York: McGraw-Hill, – 2010. – 895 p.

Leung H., White L. Insights into regression testing // Proceedings of the Conference on Software Maintenance, – 1989. – P. 60–69.

Grindal M., Offutt J., Mellin J. On the testing maturity of software producing organizations // In TAIC-PART: Proceedings of the Testing: Academic & Industrial Conference on Practice And Research Techniques. – 2006. – P. 171–180.

Elbaum S., Kallakuri P., Malishevsky A. G., Rothermel G., Kanduri S. Understanding the Effects of Changes on the Cost- Effectiveness of

Regression Testing Techniques // Journal of Software Testing, Verification, and Reliability. – 2003. – Vol. 13, N 2 – P. 65–83.

Методи та засоби програмної інженерії 144

Rothermel G., Harrold M. Selecting tests and identifying test coverage requirements for modified software // Proceedings of the International Symposium on Software Testing and Analysis. – 1994. – P. 169–184.

Elbaum S., Malishevsky A., G.Rothermel G. Prioritizing test cases for regression testing // Proceedings of the International Symposium on

Software Testing and Analysis. – 2000. – P. 102–112.

Elbaum S., Malishevsky A.G., G.Rothermel G. Test case prioritization: A family empirical studies // IEEE Transactions on Software

Engineering. – 2002. – Vol. 28, N 2. P. 159–182.

Harrold M., Soffa M. Interprocedural data flow testing // Proceedings of the ACM SIGSOFT third symposium on Software testing, analysis, and verification. – 1989. – P. 158–167.

Agrawal H., Horgan J., Krauser E., London S. Incremental regression testing // International Conference on Software Maintenance. – 1993. – P. 348–357.

Leung H., White L. A firewall concept for both control-flow and data-flow in regression integration testing // Proceedings of the Conference on Software Maintenance. – 1992. – P. 262–270.

Chen Y., Rosenblum D., Vo K. TestTube: A system for selective regression testing // Proceedings of the 16th International Conference on Software Engineering. –1994. – P. 211–222.

Vokolos F., Frankl P. A regression test selection tool based on textual differencing // Proceedings of the 3rd International Conference on Reliability, Quality & Safety of Software-Intensive Systems. –1997. – P. 3–21.

Frankl P., Rothermel G., Sayre K., Vokolos F. An empirical comparison of two safe regression test selection techniques. Proceedings of the 2003 International Symposium on Empirical Software Engineering. – 2003. – P. 195–204.

Laski J., Szermer W. Identification of program modifications and its applications in software maintenance // Proceedings of the Conference on Software Maintenance. – 1992. – P. 282–290.

Rothermel G., Harrold M. A safe, efficient regression test selection technique // ACM Transactions on Software Engineering and Methodology. – 1997. – Vol. 6, N2. – P. 173–210.

Ball T. On the limit of control flow analysis for regression test selection // ISSTA: Proceedings of the 1998 ACM SIGSOFT international symposium on Software testing and analysis. – 1998. – P. 134–142.

McGregor J., Sykes D. A Practical Guide to Testing Object-Oriented Software. Addison-Wesley. – 2001. – 416 p.

Kung D., Gao J., Hsia P., Wen F., Toyoshima Y., Chen C. On regression testing of object oriented programs // Journal of Systems and Software. –1996. – Vol. 32, N1. – P. 21–40.

Jang Y., Munro M., Kwon Y. An improved method of selecting regression tests for C++ programs // Journal of Software Maintenance: Research and Practice. – 2001. – Vol. 13, N5. – P. 331–350.

Rothermel G., Harrold M. Selecting regression tests for object-oriented software // International Conference on Software Maintenance. – March 1994. – P. 14–25.

Rothermel G., Harrold M., Dedhia J. Regression test selection for C++ software // Software Testing, Verification and Reliability. – 2000. –

Vol. 10, N2. – P. 77–109.

Harrold M., Jones J., Li T., Liang D., Orso A., Pennings M., Sinha S., Spoon S. A., Gujarathi A. Regression test selection for Java software //

Proceedings of the 16th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages and Applications, January 2001. –

P. 312–326.

Orso A., Shi N., Harrold M. Scaling regression testing to large software systems // Proceedings of the 12th ACM SIGSOFT Twelfth International Symposium on Foundations of Software Engineering. – 2004. – P. 241–251.

Mansour N., Statieh W. Regression test selection for C# programs // Advances in Software Engineering. – 2009. – P. 1–10.

Briand L., Labiche Y., Soccar G. Automating impact analysis and regression test selection based on UML designs Proceedings of the International Conference on Software Maintenance. – 2002. – P. 252–261.

Briand L., Labiche Y., He S. Automating regression test selection based on UML designs // Information and Software Technology. – 2009. – Vol. 51, N9. – P. 16–30.

Ali A., Nadeem A., Iqbal Z., Usman M. Regression testing based on UML design models // Proceedings of the 13th Pacific Rim International

Symposium on Dependable Computing. – 2007. – P. 85–88.

Farooq Q., Iqbal M., Malik Z., Riebisch M. A model-based regression testing approach for evolving software systems with flexible tool support. In 17th IEEE International Conference on Engineering of Computer-Based Systems. – March 2010. – P. 41–49.

Naslavsky L., Richardson D. Using traceability to support model-based regression testing // Proceedings of the twenty-second IEEE/ACM

international conference on Automated software engineering, ASE. November 2007. – P. 567–570.

Gorthi, R. Pasala A., Chanduka K., Leong B. Specification-based approach to select regression test suite to validate changed software // Proceedings of 15th Asia-Pacific Software Engineering Conference. – 2008. – P. 153–160.

Chen Y., Probert R., Sims D. Specification based regression test selection with risk analysis // Proceedings of the 2002 conference of the Centre for Advanced Studies on Collaborative research. – 2002. – P. 322–323.

Chittimalli P., Harrold M. Regression test selection on system requirements // Proceedings of the 1st conference on India software engineering conference, August 2008. – P. 87–96.

Orso A., Harrold M., Rosenblum D. Component metadata for software engineering tasks // Revised Papers from the Second International Workshop on Engineering Distributed Objects, EDO. Springer-Verlag. – 2000. – P. 129–144.

Orso A., Harrold M., Rosenblum D., Rothermel G., Soffa M., and Do H. Using component metacontent to support the regression testing of

component-based software Proceedings of the IEEE International Conference on Software Maintenance, November 2001. – P. 716–725.

Mao C., Lu Y. Regression testing for component-based software systems by enhancing change information // APSEC: Proceedings of the 12th Asia-Pacific Software Engineering Conference. IEEE Computer Society, December 2005. – P. 611–618.

Mao C., Lu Y., Zhang J. Regression testing for component-based software via built-in test design // Proceedings of the 2007 ACM symposium on Applied computing. – 2007. – P. 1416–1421.

Sajeev A., Wibowo B. Regression test selection based on version changes of components // Proceedings of the Tenth Asia-Pacific Software Engineering Conference Software Engineering Conference, December 2003. – 78 p.

Wu Y., Offutt J. Maintaining evolving component-based software with UML // Proceedings of 7th European Conference on Software Maintenance and Reengineering, March 2003. – P. 133–142.

Pasala A., Fung Y., Akladios F., Raju A., Gorthi R. Selection of regression test suite to validate software applications upon deployment of upgrades // 19th Australian Conference on Software Engineering, March 2008. – P. 130–138.

Zheng J., Robinson B., Williams L., Smiley K. Applying regression test selection for COTS based applications // ICSE: Proceedings of the 28th international conference on Software engineering, May 2006. – P. 512–522.

Zhao J., Xie T., Li N. Towards regression test selection for AspectJ programs // Proceedings of the 2nd workshop on Testing aspect-oriented programs. – 2006. – P. 21–26.

Xu G., Rountev A. Regression test selection for AspectJ software // Proceedings of the 29th international conference on Software Engineering, December 2007. – P. 65–74.

Haftman F., Kossmann D., Lo E. A framework for efficient regression tests on database applications // The VLDB Journal. – 2007. – Vol. 10,

N 2. – P. 145–164.

Haraty R., Mansour N., Daou B. Advanced Topics in Database Research. Idea Group. – 2004. – Vol. 3. – P. 141–165.

Методи та засоби програмної інженерії 145

Willmor D. and Embury S. A safe regression test selection technique for database-driven applications // Proceedings of the 21st IEEE International Conference on Software Maintenance.– 2005. – P. 421–430.

Xu L., Xu B, Chen Z., Jiang J., Chen H. Regression testing for web applications based on slicing // Proceedings of the 27th Annual International Computer Software and Applications Conference. – 2003. – P. 652–656.

Tarhini A., Fouchal H., Mansour N. Regression testing web services-based applications // AICCSA Proceedings of the IEEE International

Conference on Computer Systems and Applications. – 2006. – P. 163–170.

Ruth M., Tu S. A safe regression test selection technique for web services // Proceedings of the Second International Conference on Internet and Web Applications and Services. – 2007. – P. 47.

Lin F., Ruth M., Tu S. Applying safe regression test selection techniques to Java web services // International Conference on Next Generation Web Services Practices. – 2006. – P. 133–142.

Rothermel G., Untch R., Chu C., Harrold M.J. Test case prioritization: An empirical study // Proceedings of International Conference Software Maintenance, August 1999. – P. 179–188.

Harrold M. J., Larsen L., Lloyd J., Nedven D., Page M., Rothermel G., Manvinder S., Smith M. Aristotle: A System for Development of Program Analysis Based Tools // Proceedings of the 33rd annual on Southeast regional conference. – 1995. – P. 110–119.

Srivastava A. Thiagarajan J. Effectively prioritizing tests in development environment // Proceedings of the International Symposium on Software Testing and Analysis. – 2002. – P. 97–106.

Belli F., Eminov M., Gokco N. Coverage-Oriented, Prioritized Testing-A Fuzzy Clustering Approach and Case Stud // Proceedings of the Third

Latin-American conference on Dependable Computing. – 2007. – P. 95–110.

Do H., Mirarab S., Tahvildari L., Rothermel G. An Empirical Study of the effect of time constraints on the cost benefits of regression testing // Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering. – 2008. – P. 71–82.

Jiang B., Zhang Z., Chan W.K., Tse T.H., Adaptive Random test case prioritization // Proceedings of International Conference on Automated

Software Engineering. – 2009. – P. 233–243.

Bryce R.C., Sampath S., Memon A.M. Developing a Single Model and Test Prioritization Strategies for Event Driven Software // Transactions on Software Engineering. – 2010. – P. 48–63.

Wong W.E., Horgan J.R., London S., Aggarwal A. A study of effective regression testing in practice // Proceedings of the Eighth International

Symposium Software Reliability Engineering, November 1997. – P. 230–238.

Korel B., Tahat L., Harman M., Test Prioritization Using System Models // The Proceedings of 21st International Conference on Software Maintenance. – 2005. – P. 247–256.

Korel B., Koutsogiannakis G., Talat L.H. Modelbased test suite prioritization Heuristic Methods and Their Evaluation // Proceedings of 3rd workshop on Advances in model based testing. – 2007. – P. 34–43.

Filho R.S.S., Budnik C.J., Hasling W.M., Kenna M.M., Subramanyam R. Supporting concern based regression testing and prioritization in a

model driven environment // Proceedings of 34th Annual Computer software and Applications conference Workshops. – 2010. – P. 323–328.

Srikanth H., Williams L., Osborne J. System Test Case Prioritization of New and Regressionb Test Cases // Proceedings of International Symposium on Empirical Software Engineering. – 2005. – P. 64–73.

Krishnamoorthi R., Mary S.A. Incorporating varying requirement priorities and costs in test case prioritization for new and regression testing // Proceedings of International Conference on Computing, Communication and Networking. – 2008. – P. 1–9.

Krishnamoorthi R., Mart S.A. Factor oriented requirement coverage based system test case prioritization of new and regression test cases // Journal of information and software technology. – 2009. – Vol. 51. – P. 799–808.

Kim J.M., Porter A. A History-Based Test Prioritization Technique for Regression Testing in Resource Constrained Environment // Proceedings of the 24th International Conference Software Engineering, May 2002. – P. 119–129.

Fazlalizadeh Y., Khalilian A., Azgomi H.A., Parsa S. Incorporating historical test case performance data and resource constraints into test case prioritization // Lecture notes in Computer Science, Springer. – 2009. – P. 43–57.

Walcott K.R., Soffa M.L., Kapfhammer G.M., Roos R.S. Time aware test suite Prioritization // Proceedings of International Symposium on software Testing and Analysis, July 2006. – P. 1–12.

Conrad A.P., Roos R. S. Empirically Studying the role of selection operators during search based test suite prioritization // Proceedings of the ACM SIGEVO Genetic and Evolutionary Computation Conference, Portland, Oregon. – 2010. – P. 1373–1380

Smith A.M., Kapfhammer G.M. An empirical study of incorporating cost into test suite reduction and prioritization // Proceedings of ACM

Symposium on Applied Computing. – 2009. – P. 461–467.

Yoo S., Harman M. Pareto efficient multi-objective test case selection // Proceedings of the 2007 International Symposium on Software Testing and Analysis. – 2007. – P. 140–150.

The Mathworks, Inc. MATLAB. Website, April 2011. http://www.mathworks.com.

Jennings N., Wooldridge M. Agent-Oriented Software Engineering // Artificial intelligence. – 2000. – Vol. 117. – Р. 277–296.


Refbacks

  • There are currently no refbacks.