CPN simulation-based test case generation from natural language requirements

Abstract

Software Engineering faces challenges such as difficulty in understanding the user needs, ambiguous specifications, poorly defined requirements and therefore problems in interpreting the system requirements. Model-based testing (MBT) is presented as an alternative for solving these problems by using (semi-)formal methods, in the specification, modelling or analysis of requirements, as well as by automatically generating test cases. This work proposes and implements a test generation strategy from Natural Language (NL) requirements via translation into Coloured Petri Nets (CPN), an extension of Petri Nets that supports model structuring. This strategy extends previous work on the NAT2TEST framework, which involves syntactic and semantic analyses of NL requirements and the generation of Data-Flow Reactive System (DFRS) as an intermediate representation, from which target formal models can be obtained for the purpose of test case generation. Our contributions include a systematic translation of DFRSs into CPN models, besides a strategy for test generation. We illustrate our overall approach with a running example. Therefore, this work presents a variant for the NATural Language Requirements to TEST Cases (NAT2TEST) strategy, in which the Coloured Petri Nets (CPN) is used as an intermediate model. The NAT2TEST strategy, which is applicable to discrete or continuous systems, consists of five phases: syntactic and semantic analyses, DFRS generation, CPN generation, and generation of test cases. The approach proposed here, which is based on Petri Nets simulation, has as benefit the maturity of the theory and tools related to CPN. Moreover, there are available resources for analysing structural and behavioural properties of the model. The process is automated by the NAT2TEST tool until the DFRS generation. The model translation from the DFRS to the CPN is automated by Spoofax framework. Finally, the test cases generation occurs automatically via simulations held in the CPN Tools. Our strategy was evaluated considering examples from the literature (Vending Machine and Nuclear Power Plant) and the aerospace industry (Priority Control). We analysed performance and the ability to detect defects generated via mutation. In general, our strategy outperformed the considered baseline: random testing. We also compared our strategy with the CSP version.