Metamorphic Relation Patterns for Query-Based Systems

Abstract

Searching and displaying data based on user queries is a key feature of most software applications such as information systems, web portals, web APIs, and data analytic platforms. The large volume of data managed by these types of systems, henceforth called query-based systems (QBS), makes them extremely hard to test due to the difficulty to assess whether the output of a query is correct, the so-called oracle problem. Metamorphic testing has proved to be a very effective approach to alleviate the oracle problem in QBS, enabling the detection of bugs in data repositories, large e-commerce sites, and some of the most used software applications worldwide such as Google Search and YouTube. We have observed, however, that the metamorphic relations used to test different types of QBS are very similar, regardless of their domain, since all of them exploit standard query features such as filtering and ordering. Inspired by this finding, in this paper we present a catalogue of metamorphic relation patterns to assist testers in the identification and inference of metamorphic relations in QBS. For the definition of the patterns we resorted to the root of most query languages: relational algebra. We show how the proposed patterns help in the identification of metamorphic relations in the e-commerce platform PrestaShop, the email service Gmail, and the mobile application of video streaming HBO.