With the benefits of reducing time-cost and human efforts, automated testing has been widely used for quality assurance of mobile applications (apps). However, in complex interactive activities, manual testing can achieve higher coverage. However, the effectiveness of manual testing is highly dependent on the vital User Operation Process (UOP) of experienced testers.
To solve the problems, a research team led by Zhenyu CHEN and Chunrong FANG published their new research on 15 Oct 2023 in Frontiers of Computer Science co-published by Higher Education Press and Springer Nature.
We propose an Iterative Android Automated Testing (IAAT) method that automatically records, extracts, and integrates UOPs. The UOP from manual testing will be iteratively introduced in the test logic of the automated testing tool. The test results are further fed back to testers to achieve higher coverage in each iteration. We conducted experiments on 10 well-known mobile apps and 50 UOPs to compare IAAT with Monkey and the original automated test. According to the experimental results, IAAT shows a significant improvement in the IAAT compared with the test logic without introducing human knowledge.
In the research, IAAT is designed to increase the coverage rate to the greatest extent. To achieve this purpose, we create two-layer iterations in IAAT. One of the iterations is within the automated testing, and the other is between the UOP and the automated testing. The iteration inside the automated testing is to improve the coverage by performing multiple iterations, which is different from the traditional one-time testing process. Each automated test's starting point refers to the node not covered by the previous round of testing. It is essentially equivalent to running automated testing based on the prior round of testing to obtain higher test coverage. We have realized the IAAT method in a practical tool. Our practical tool, IAAT, separates test logic (using DFS in this paper) from UOP extraction and fusion, allowing it to be combined with different test logic tools for even greater coverage.
IAAT brings human knowledge to efficient automated testing and integrates the above two advantages to complete the test. Specifically, IAAT is divided into two main steps:
Step 1, Complete the UOP fusion and generate formatted UOPs to match the test logic of the automated testing tool. The UOP information can be provided in two ways, either by manual provisioning (e.g., from manual clicks or test scripts written by testers) or automatically extracted from execution traces in IAAT. Step 2, Combine formatted UOP and automated testing tools for testing. The IAAT integrates the existing UOP to generate a coverage tree. It describes the activity information currently covered about the application under test and guides the tool to start a new round of automated testing accordingly. In the next round of testing, the Widget that was not covered in the previous round will be overridden first, then run the testing tool iteratively until the Activity overlay tree is completely traversed.
In the future, we will try to introduce UOPs into the testing logic of more mature tools to help them break the original coverage bottleneck and cross more complex Widgets thus achieving high coverage of apps.
