WEB PAGE ELEMENT IDENTIFICATION USING SELENIUM AND CNN: A NOVEL APPROACH

Abstract

Web applications have become an integral aspect of our daily lives, thus rendering efficient testing and validation imperative. The process of software testing holds significant importance within the entirety of software development. In the present day, there exists a plethora of automated software testing tools that cater to the examination of diverse software applications, be it desktop-based, mobile applications, or web-based applications (Prasad et al., 2020). Within this discourse, we propose a novel approach that combines the prowess of Selenium, a renowned web automation tool, with convolutional neural networks (CNN), in order to automatically identify elements of web pages. Traditional web scraping methods often encounter difficulties when faced with dynamically generated content and intricate web structures. Our method, on the other hand, utilizes Selenium to navigate web pages, manipulate elements, as well as capture screenshots. These screenshots are subsequently processed using CNN to ascertain the identification and classification of web page elements.

To commence, we shall provide an elaborate explanation of the capabilities possessed by the Selenium framework, with a specific focus on its capacity to automate user interaction across various browsers when it comes to web applications. Selenium effectively captures real-time screenshots of web pages that are visible to users, thereby creating a valuable dataset that aids in the identification of elements. Subsequently, we shall delve deeper into the architecture of CNN and elucidate the means through which features can be extracted from these screenshots, enabling the classification of elements such as buttons, text boxes, and images. In order to validate our approach, experiments were conducted on diverse websites that possess intricate layouts and dynamic content. The outcome of these experiments serves as a testament to the efficacy of our method in accurately identifying and classifying web page elements, even in the most demanding of scenarios. Our research contributes significantly to the realms of web testing and automation by furnishing a robust solution for the identification of web page images. This approach serves to enhance the efficiency of the web testing process, minimize the need for manual intervention, and guarantee the dependability of web applications. As web technology continues to progress, our method serves as a promising avenue for automated website analysis and quality assurance.