TAROT -- 16th International Summer School on Training And Research On Testing

TAROT (Training And Research On Testing) is a network created to foster the mobility of students, faculty members and research scientists working in the field of testing of software and communication systems. This summer school brings together lecturers, researchers, students and people from the industry across Europe for one week of presentations, discussions and getting to know each other.

The main goal of the TAROT Summer School is to give researchers and particularly Ph.D. students the opportunity to follow a number of tutorials or invited talks by key experts in the field. The TAROT Summer School is open to researchers from any institution in the world, working in the area of testing, both from academia and industry.


General Chair

Ana PaivaFaculty of Engineering of the University of Porto, Portugal

Ana CavalliTelecom & Management SudParis, France

Anna Rita FasolinoUniversitá Degli Studi Di Napoli Federico II, Italy

Bernhard Aichernig Graz University of Technology, Austria

Eda MarchettiCNR (the Italian National Research Council), in Pisa, Italy

Fatiha ZaïdiParis Sud University, France

Francisco Palomo-LozanoUniversity of Cádiz, Spain

José Creissac CamposUniversidade do Minho, Portugal

Paolo ArcainiNational Institure of Informatics, Japan

Pascoal FariaFaculty of Engineering of the University of Porto, Portugal

Rob HieronsUniversity of Sheffield, UK

Rui AbreuIST, University of Lisbon, Portugal

Tanja E.J. VosUniversidad Politecnica de Valencia, Spain

Pablo C. CañizaresComplutense University of Madrid, Spain

Manuel NúñezComplutense University of Madrid, Spain

We will speak about

Automated functional testing
of mobile applications

Anna Rita FasolinoUniversitá Degli Studi Di Napoli Federico II, Italy


Bernhard AichernigGraz University of Technology, Austria

Software Test
& Mobile Testing

Ana Paiva Faculty of Engineering of the University of Porto, Portugal

Framework for attacks tolerance.
Application to web services.

Ana Cavalli and Fatiha ZaïdiTELECOM SudParis and Paris Sud University, France

Francisco Palomo-LozanoUniversity of Cádiz, Spain

Rob HieronsUniversity of Sheffield, UK

Paolo ArcainiNational Institute of Informatics, Japan

Rui AbreuIST, University of Lisbon, Portugal

José CamposUniversidade do Minho, Portugal

Pablo C. Cañizares and Manuel NúñezComplutense University of Madrid, Spain

Summary of the talks

Learning-Based Testing Testing has always been a challenge due to (1) its incompleteness by nature, (2) the lack of good specifications and (3) by its high demand for resources. With the growing complexity of the systems-under-test the situation is not likely to improve. The combination of automata learning with model-based testing offers opportunities to further increase the level of automation in testing. In my talk I will introduce this line of research and report about our recent results including applications in the Internet of Things and autonomous driving. We will discuss the learning and testing of several classes of models/systems including Mealy machines, Markov decision processes and timed automata.

Engineering safer interactive computing systems We interact with technology to monitor and control the most varied aspects of our lives. Particularly for safety technology, such as is the case of medical devices, badly designed user interfaces can have unacceptable costs. Formal methods, in particular when tool supported, are gaining acceptance as a means to perform rigorous analysis of complex systems. This talk will provide an overview of work on interactive computing system modelling and analysis, discussing also its integration with more traditional user centred development approaches. Examples of problems with real user interfaces will be presented and the role that formal analysis can play discussed.

Automated functional testing of mobile apps This talk presents the state of the art in the field of automated functional testing of mobile applications based on the results of a systematic mapping study of the current literature. A classification of existing approaches of mobile application functional testing on the basis of the offered support to testing automation and addressed testing levels will be provided. Moreover, the talk will report how the proposed techniques and tools have been evaluated in terms of the characteristics of the experiments carried out, the involved applications under test, and the performed comparisons with other techniques and tools. Eventually the talk will discuss the current research trends, the existing gaps emerged from the study, and the open issues and future trends in this testing field.

A Review of Test-suite-based Automatic Program Repair Techniques This talk focuses our recent work on test-suite-based automatic program repair, where performed a large-scale experiment using 11 Java test-suite-based repair tools and 5 benchmarks of bugs. The main goal was to have a better understanding of the current state of automatic program repair tools, and our investigation was guided by the hypothesis that the repairability of repair tools might not be generalized across different benchmarks. Moreover, we will discuss a repair execution framework, named RepairThemAll, that adds an abstraction around repair tools and benchmarks, which can be further extended to support additional repair tools and benchmarks.

Framework for attacks tolerance. Application to web services. Web services allow the communication of heterogeneous systems on the Web. These facilities make them particularly suitable for deploying in the cloud. Although research on formalization and verification has improved trust in Web services, issues such security and attacks tolerance are not fully addressed. In this lecture, we will present a formal methodology that it is based on three techniques modelling, diversification and reflection. To conclude we will propose a formal Web service testing framework by incorporating these complementary mechanisms in order to take advantage of the benefits of each of them.

Testing for Automatic Repair of Models Model-based testing (MBT) uses models for different testing activities, such as test case generation. However, often models are incomplete, wrong, or outdated, and so they do not precisely represent the reference implementation acting as oracle. In these cases, models need to be updated in order to be used for MBT. Testing approaches can easily spot differences between the model and the oracle; however, it remains the problem of how to modify the model to align it with the oracle. This talk presents approaches that solve this problem by combining testing with automatic repair, using either mutation or abstraction as a means for repair. Target notations of these approaches are variability models, regular expressions, and timed automata.

Enhancing Authorization Systems with Data Protection Compliance The General Data Protection Regulation (GDPR) defines the principle of Integrity and Confidentiality, and implicitly calls for the adoption of Access Control Systems for regulating the access to personal data. For this an essential step is to augment access control systems with concepts from the GDPR, and encode the most suitable provisions of the GDPR as Access Control Policies (ACPs). This requires organizations to deploy and test adequate fine-grained access control mechanisms. This talk explores the available proposals for the automatic enforcing and testing of the data protection regulations and discusses their main challenges.

Optimal Product Selection from Feature Models A feature model (FM) specifies the sets of features that define valid products in a software product line. This talk explores the many-objective optimisation problem of choosing optimal products from a FM based on user preferences. This problem has been found to be difficult for a purely search-based approach, leading to classical many-objective optimisation algorithms being enhanced by either adding in a valid product as a seed or by introducing additional mutation and replacement operators that use a SAT solver. This talk describes the SIP method that instead enhances the search in two ways: by providing a novel representation and also by optimising first on the number of constraints that hold and only then on the other objectives.

Formal verification with ACL2 When testing is not enough, formal verification comes to the rescue. ACL2 is a Lisp-based programming language, a computational logic, and an automated reasoning system for modelling, specifying and formally verifying computational systems. ACL2 is industrial-strength and has been successful in a number of real-world projects since the early 1990s. Unlike other similar systems, ACL2 is based on a simple logic, which it compensates with higher automation. We will present a brief history of formal verification, introduce different aspects of ACL2, work out examples, and discuss some challenges.

Metamorphic Testing of Cloud Systems Cloud systems are supported by complex infrastructures, consisting of a wide diversity of subsystems and components. The heterogeneous nature of these systems, their size, the high number of users that concurrently request services, and the virtualisation, among other factors, hamper their validation. Unfortunately, it is not feasible to use conventional testing methods to evaluate the correctness of cloud systems. This talk presents a set of methodologies and techniques to model, validate and optimise cloud systems in a viable and effective way. Our work uses Metamorphic Testing, in combination with Mutation Testing, to analyse the effectiveness of these properties to detect anomalous behaviours of the system.