SIT: Sampling-based interactive testing for self-adaptive apps

摘要

Self-adaptive apps contain adaptation logic which decides methods of delivering services based on environmental conditions.. These app are usually meant for embedded systems or smart phones and their sensors collect information on environmental conditions. The apps have an interaction loop that connects to its running environment and adaptive logic makes changes according to the sensed changes. Though self-adaptive apps offer flexible services, they have to address environmental uncertainty. Developing these applications is not easy and is error prone. A self-adaptive app executes in an infinite reaction loop with its environment and as the environment changes it has to make decisions based on an infinite series of input/output pairs, which may not be independent. Testing of self-adaptive apps is not straight forward since it must consider the environment. This article uses iteration to denote one pass in executing an app's reaction loop. A self-adaptive app interacts with its environment in an uncertain way called internal and external uncertainty. The former is related to the adaptation algorithm. This article is concerned with external uncertainty where the app understands the environment and makes normal or faulty decisions. While traditional program bugs are related to the software, a self-adaptive app's bugs depend on the program, its running environment and uncertainty in the interaction between the app and its environment. The bugs can be present in any one of the three. This uniqueness makes the test of self-adaptive apps more challenging. An app can fail after its multiple executions of its reaction loop with its environment. These executions are different iterations with different input/output pairs. Thus testing by taking in to account is a difficult task and the possibility of missing should not be overruled. Many of the existing approaches for testing under these uncertainties have a drawback of missing about 30% of the bugs or more. This study proposes an ideal approach SIT for testing self-adaptive apps. The test consists of an interactive application model and a test generation method. This approach is light-weight in the sense that it samples only those inputs that require exploration. This method improves bug detection by 22.4% to 42.2% with reduced cost. (50 refs.)