Traditional functional testing mainly validates software quality against functional requirements. It does not validate software quality against code flexibility, which is regarded as a non-functional requirement. While such validation is important, we have lacked measurable criteria. As such, code flexibility has rarely been systematically tested. Recently, design patterns have been adopted in object-oriented development as a recommended artifact for enhancing potentials in software reuse and evolution, in that each design pattern presents a non-hardcoding solution intended to achieve certain degree of code flexibility. However, the desired code flexibility may not be achieved upon inappropriate utilization of design patterns, particularly when multiple patterns are utilized. In this re...[ Read more ]

Traditional functional testing mainly validates software quality against functional requirements. It does not validate software quality against code flexibility, which is regarded as a non-functional requirement. While such validation is important, we have lacked measurable criteria. As such, code flexibility has rarely been systematically tested. Recently, design patterns have been adopted in object-oriented development as a recommended artifact for enhancing potentials in software reuse and evolution, in that each design pattern presents a non-hardcoding solution intended to achieve certain degree of code flexibility. However, the desired code flexibility may not be achieved upon inappropriate utilization of design patterns, particularly when multiple patterns are utilized. In this research, we propose a methodology referred to as pattern conformance testing, which validates the flexibility of an implementation against the conformance requirements of a pre-announced set of design patterns. We also propose how the solution structures of design patterns can be enhanced to become the measurable criteria so that test oracles can be designed. No similar work has been found in the literature.

Our methodology consists of a test model and a test strategy. A test model represents the elements of code implementing design patterns, and the criteria that characterize a set of design patterns. A test strategy is a mechanism containing test cases to evaluate the code flexibility based on a test model and the source code. The methodology will firstly use the test model to derive the variability of elements of an implementation. Then the test strategy will be enacted to check if the observed results are consistent with the derived ones. Given that the design of a real-life system may need to employ multiple patterns, an interesting part of this research is to support the validation of code flexibility on the implementation with intertwining logical code structures where some methods participate in more than one application of pattern.

To evaluate the feasibility of pattern conformance testing, we applied it to four Java programs with different designs but the same functional and flexibility requirements, as well as comparing their test results. It successfully revealed the implementation faults from achieving the intended code flexibility. Since these faults do not lead to program incorrectness, they could not be revealed by traditional functional tests.