UC Irvine

For 15% of the world population with disabilities, accessibility is arguably the most critical software quality attribute. The ever-growing reliance of users with disability on mobile apps further underscores the need for accessible software in this domain. Manual accessibility testing with assistive services is a high-fidelity form of testing; however, it is time-consuming and requires deep knowledge of various aspects of accessibility. Existing automated accessibility assessment techniques primarily aim to detect violations of predefined guidelines, thereby often overlook the way software is actually used by users with disability, i.e., with assistive services. Since disabled users, especially the ones with motor or visual impairments, are heavily reliant on assistive services in interacting with apps, many important cues are missed when these services are not considered in the evaluation of an app’s accessibility.

This dissertation proposes a three-pronged approach to advance accessibility testing for mobile applications by including assistive services in the evaluation process. In the first prong, I introduce a new technique to extract main use cases of the software from the existing software tests, then re-execute them from the standpoint of users with disabilities with various assistive services. In the second prong, I introduce a completely automated way to crawl applications using assistive services to detect accessibility issues at runtime. Finally, in the third prong, I introduce a semi-automated technique to aid manual accessibility testers to efficiently evaluate applications with various assistive services. To show the feasibility of these ideas, this dissertation particularly presents and proposes automated and semi-automated tools implemented for Android platform, namely for each prong (1) LATTE, (2) GROUNDHOG, OVERSIGHT, and (3) A11YPUPPETRY.

All conducted experiments on real-world subject apps corroborate the effectiveness and efficiency of the proposed approaches, and their ability to detect various types of accessibility issues in mobile applications.