Model-driven architecture based testing using software architecture viewpoints

Abstract

Software testing is the process of checking whether a system meets the specifications and fulfills its intended purpose. Testing a system requires executing the test cases that can detect the potential defects in the program. In general, exhaustive testing is not possible or practical for most real programs due to the large number of possible inputs and sequences of operations. Because of the large set of possible tests only a selected set of tests can be executed within feasible time limits. As such, the key challenge of testing is how to select the tests that are most likely to expose failures in the system. Model-based testing (MBT) relies on models of system requirements and behavior to automate the generation of the test cases and their execution. Model based testing can use different representations of the system to generate testing procedures for different aspects of the software systems. Example models include finite state machines (FSMs), Petri Nets, I/O automata, and Markov Chains. A recent particular trend in MBT is to adopt architecture models to identify the defects related to systemic properties. These systemic properties are typically defined in architecture views which represent the gross level structure of the system from particular concern perspective. Assessing software system correctness with respect to architectural specifications is called architecture based testing (ABT). Many studies have focused on architecture based testing in which different models have been applied. However none of these have so far explicitly focused on adopting architecture views for deriving the test cases. In this thesis, we first provide a systematic review on existing model-driven architecture based testing. We define all the existing processes in the literature and discuss the current limitations. Based on the result of the systematic review and our own analysis we provide a novel model-driven architecture based testing approach using architecture views. With the approach we focus on detecting the deviations in the code from the architectural views. For this we use models of architecture views together with executable transformation model to generate the test cases which are then executed on the real code. Our approach has been evaluated within a real industrial context of The Scientific and Technological Research Council of Turkey Software Technologies Institute (STRCT-STI). The results of the industrial case study showed that model-driven architecture based testing can be effective for reducing the time to generate and execute the test cases, and enhancing the reliability of the system.