ARTICLE DOWNLOAD

How does combinatorial testing perform in the real world: an empirical study

10.00$
ARTICLE DOWNLOAD

How does combinatorial testing perform in the real world: an empirical study

10.00$

Linghuan Hu, W. Eric Wong, D. Richard Kuhn & Raghu N. Kacker 

Abstract

Studies have shown that combinatorial testing (CT) can be effective for detecting faults in software systems. By focusing on the interactions between different factors of a system, CT shows its potential for detecting faults, especially those that can be revealed only by the specific combinations of values of multiple factors (multi-factor faults). However, is CT practical enough to be applied in the industry? Can it be more effective than other industry-favored techniques? Are there any challenges when applying CT in practice? These research questions remain in the context of industrial settings. In this paper, we present an empirical study of CT on five industrial systems with real faults. The details of the input space model (ISM) construction, such as factor identification and value assignment, are included. We compared the faults detected by CT with those detected by the in-house testing teams using other methods, and the results suggest that despite some challenges, CT is an effective technique to detect real faults, especially multi-factor faults, of software systems in industrial settings. Observations and lessons learned are provided to further improve the fault detection effectiveness and overcome various challenges.

Only units of this product remain
Year 2020
Language English
Format PDF
DOI 10.1007/s10664-019-09799-2