Automated Generation of Metamorphic Relations for Query-Based Systems

Abstract

Searching and displaying data based on user queries is a pervasive feature of most software applications such as information systems, web portals, and web APIs. The large volume of data managed by these types of systems, henceforth called query-based systems (QBSs), makes them extremely hard to test due to the difficulty to assess whether the output of a query is correct, the so-called test oracle problem. Metamorphic testing has proved to be a very effec tive approach to alleviate the oracle problem in QBSs, by exploiting the relations among multiple executions of the QBS under test, so called metamorphic relations (MRs). However, the identification of MRs mostly remains a manual and creative task, limiting the appli cability of the approach. In this paper, we propose a method for the automated generation of MRs in QBSs starting from a lightweight specification of the query parameters of the system. Evaluation results show that hundreds of MRs can be automatically identified in real-world systems like IMDb, SkyScanner, or YouTube in just a few seconds