EVALUATING REGRESSION TESTING PERFORMANCE THROUGH MACHINE LEARNING FOR TEST CASE REDUCTION
Keywords:
Regression Testing, Prioritized Test, Reinforcement Learning, Software. Machine, Learning, Test Case, Software Testing, Performance TestingAbstract
With regression testing, you can be confident that any modifications you made to the software's code didn't break any pre-existing modules. The size of the test suite grows as the product evolves to meet the needs of its end users. After software changes, it's important to run all of the tests again to ensure that nothing broke. It's expensive to re-execute test cases, especially when new cases are added to the mix. The cost of regression testing may be reduced by using strategies suggested by the scientific community to prioritize the test suite. Prioritizing a test suite should aim to optimize fault detection with few test cases. By eliminating less important tests, "test suite minimization" helps keep test suite sizes down. We offer a four-step approach to prioritizing test suites using reinforcement learning in this research. As a first step, the suggested interaction recording technologies for the android app are used to compile test log and user datasets. The first stage involves gathering data, and the second involves using the suggested reinforcement learning model to predict the greatest future reward sequence list. Finally, the suggested prioritizing method represents the ordered set of tests. In the end, the fault seeding method is utilized to verify the findings of software engineering professionals. Five different case studies are used to assess the effectiveness of the proposed reinforcement learning-based test suite optimization approach. The significance of the proposed mechanism for regression testing is shown by the results of the performance assessment, which indicate that it outperforms baseline techniques based on random and t-SANT methods
References
N. Gupta, A. Sharma and M. K. Pachariya, “An insight into test case optimization: Ideas and trends with future perspectives,” IEEE Access, vol. 7, pp. 22310–22327, 2019.
G. Rothermel, R. H. Untch and M. J. Harrold, “Prioritizing test cases for regression testing,” IEEE Transactions on Software Engineering, vol. 27, no. 10, pp. 929–948, 2021
H. Jahan, Z. Feng and S. M. H. Mahmud, “Risk-based test case prioritization by correlating system methods and their associated risks,” Arabian Journal for Science and Engineering, vol. 45, no. 4, pp. 6125– 6138, 2020.
R. Huang, Q. Zhang, T. Y. Chen, J. Hamlyn-Harris, D. Towey et al., “An empirical comparison of fixed strength and mixed-strength for interaction coverage-based prioritization,” IEEE Access, vol. 6, pp. 68350– 68372, 2018.
Z. Li, M. Harman and R. M. Hierons, “Search algorithms for regression test case prioritization,” IEEE Transactions on Software Engineering, vol. 33, no. 4, pp. 225–237, 2017.
S. Elbaum, A. Malishevsky and G. Rothermel, “Incorporating varying test costs and fault severities into test case prioritization,” in Proc. of the 23rd Int. Conf. on Software Engineering. ICSE, Toronto, Ont., Canada, pp. 329–338, 2021.
A. Bajaj and O. P. Sangwan, “A systematic literature review of test case prioritization using genetic algorithms,” IEEE Access, vol. 7, pp. 126355–126375, 2019.
B. Miranda and A. Bertolino, “An assessment of operational coverage as both an adequacy and a selection criterion for operational profile-based testing,” Software Quality Journal, vol. 26, no. 4, pp. 1571–1594, 2018.
R. Matinnejad, S. Nejati, L. C. Briand and T. Bruckmann, “Test generation and test prioritization for simulink models with dynamic behavior,” IEEE Transactions on Software Engineering, vol. 45, no. 9, pp. 919–944, 2019.
M. Khatibsyarbini, M. A. Isa, D. N. A. Jawawi, H. N. A. Hamed and M. D. Mohamed Suffian, “Test case prioritization using firefly algorithm for software testing,” IEEE Access, vol. 7, pp. 132360–132373, 2019.
S. Tahvili, R. Pimentel, W. Afzal, M. Ahlberg, E. Fornander et al., “sOrTES: A supportive tool for stochastic scheduling of manual integration test cases,” IEEE Access, vol. 7, pp. 12928–12946, 2019.
R. Mukherjee and K. S. Patnaik, “Prioritizing JUnit test cases without coverage information: An optimization heuristics-based approach,” IEEE Access, vol. 7, pp. 78092–78107, 2019.
C. Lu, J. Zhong, Y. Xue, L. Feng and J. Zhang, “Ant colony system with sorting-based local search for coverage-based test case prioritization,” IEEE Transactions on Reliability, vol. 69, no. 3, pp. 1–17, 2019.
M. Abdur, M. Abu and M. Saeed, “Prioritizing dissimilar test cases in regression testing using historical failure data,” International Journal of Computer Applications, vol. 180, no. 14, pp. 1–8, 2018.
D. Hao, L. Zhang, L. Zang, Y. Wang, X. Wu et al., “To be optimal or not in test-case prioritization,” IEEE Transactions on Software Engineering, vol. 42, no. 5, pp. 490–505, 2016.
Y. Bian, Z. Li, R. Zhao and D. Gong, “Epistasis based ACO for regression test case prioritization,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 1, no. 3, pp. 213–223, 2017.
S. Eghbali and L. Tahvildari, “Test case prioritization using lexicographical ordering,” IEEE Transactions on Software Engineering, vol. 42, no. 12, pp. 1178–1195, 2016.
A. Marchetto, M. M. Islam, W. Asghar, A. Susi and G. Scanniello, “A multi-objective technique to prioritize test cases,” IEEE Transactions on Software Engineering, vol. 42, no. 10, pp. 918–940, 2016.
M. Abu Hasan, M. Abdur Rahman and M. Saeed Siddik, “Test case prioritization based on dissimilarity clustering using historical data analysis,” in International Conference on Information, Communication and Computing Technology, In: S. Kaushik, D. Gupta, L. Kharb and D. Chahal (Eds.), vol. 750, Singapore: Springer, pp. 269–281, 2017.
H. Mei, D. Hao, L. Zhang, L. Zhang, J. Zhou et al., “A static approach to prioritizing JUnit test cases,” IEEE Transactions on Software Engineering, vol. 38, no. 6, pp. 1258–1275, 2022.
M. R. N. Dobuneh, D. N. Jawawi, M. Ghazali and M. V. Malakooti, “Development test case prioritization technique in regression testing based on hybrid criteria,” in 8th Software Engineering Conf. (MySEC), Langkawi, Malaysia, pp. 301–305, 2017.
B. Jiang and W. K. Chan, “Input-based adaptive randomized test case prioritization: A local beam search approach,” Journal of Systems and Software, vol. 105, no. 2, pp. 91–106, 2015”.
B. Miranda, “FAST approaches to scalable similarity-based test case prioritization,” in IEEE/ACM 40th Int. Conf. on Software Engineering (ICSE), Gothenburg, Sweden, pp. 222–232, 2018.
A. Nanda, S. Mani, S. Sinha, M. J. Harrold and A. Orso, “Regression testing in the presence of non-code changes,” in Fourth IEEE Int. Conf. on Software Testing, Verification and Validation, Berlin, Germany, pp. 21–30, 2021.
L. Xiao, H. Miao, W. Zhuang and S. Chen, “An empirical study on clustering approach combining fault prediction for test case prioritization,” in IEEE/ACIS 16th Int. Conf. on Computer and Information Science (ICIS), Wuhan, China, pp. 815–820, 2017.
P. Konsaard and L. Ramingwong, “Total coverage-based regression test case prioritization using genetic algorithm,” in 12th Int. Conf. on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), Hua Hin, Thailand, pp. 1–6, 2015.
T. B. Noor and H. Hemmati, “Test case analytics: Mining test case traces to improve risk-driven testing,” in IEEE 1st Int. Workshop on Software Analytics (SWAN), Montreal, QC, Canada, pp. 13–16, 2015.
Abid, R.; Nadeem, A.A novel approach to multiple criteria based test case prioritization. In Proceedings of the 2017 13th International Conference on Emerging Technologies (ICET), Islamabad, Pakistan, 27–28, December 2017; IEEE: Piscataway, NJ, USA, 2022; pp.1–6.
Khan, S.U.R.; Lee, S.P.; Parizi, R.M.; Elahi, M. A code coverage-based test suite reduction and prioritization framework. In Proceedings of the 2014 Fourth World Congress on Information and Communication Technologies (WICT), Malacca, Malaysia, 8–11 December 2014; IEEE: Piscataway, NJ, USA, 2021; pp.229–234.
Nagar,R.; Kumar,A.; Singh,G.P.; Kumar,S. Test cases election and prioritization using cuckoos search algorithm. In Proceedings of the 2015 International Conference on Futuristic Trends on Computational Analysis and Knowledge Management (ABLAZE), Noida, India, 25–27 February 2015; IEEE: Piscataway, NJ, USA, 2021pp.283–288.
Spieker, H.; Gotlieb, A.; Marijan, D.; Mossige, M. Reinforcement learning for automatic test case prioritization and selection in continuous integration. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, Santa Barbara, CA, USA,10–14June2017; ACM: NewYork, NY, USA, 2020; pp.12–22.
Chen, J.; Zhu, L.; Chen, T.Y.; Towey, D.; Kuo, F.C.; Huang, R.; Guo, Y. Test case prioritization for object-oriented software: An adaptive random sequence approach based on clustering.J.Syst.Softw.2018,135,107–125
D. Marijan, M. Liaaen, A. Gotlieb, S. Sen and C. Ieva, “TITAN: Test suite optimization for highly configurable software,” in IEEE Int. Conf. on Software Testing, Verification and Validation (ICST), Tokyo, Japan, pp. 524–531, 2017.
R. Lachmann, “Machine learning-driven test case prioritization approaches for black-box software testing,” in The European Test and Telemetry Conf., Von-Münchhausen-Str 49 31515 Wunstorf Ger, AMA Serv. GmbH, 2018.
A. Ansari, A. Khan, A. Khan and K. Mukadam, “Optimized regression test using test case prioritization,” Procedia Computer Science, vol. 79, pp. 152–160, 2016.
