Database Management System (DBMS) plays a core role in modern software with varieties of disciplines from mobile apps to online banking systems. It is critical that the DBMSs provide correct data to all applications. When the DBMS returns incorrect data, a correctness bug is triggered. Current production level DBMSs still suffer from insufficient testing due to the limited hand-written test cases. Recently several works proposed to automatically generate many test cases with query transformation, a process of generating an equivalent query pair and testing a DBMS by checking whether the system returns the same result set for the query pair. However, all of the existing works still heavily rely on manual work to provide a transformation which largely confines their exploration of the val...[ Read more ]

Database Management System (DBMS) plays a core role in modern software with varieties of disciplines from mobile apps to online banking systems. It is critical that the DBMSs provide correct data to all applications. When the DBMS returns incorrect data, a correctness bug is triggered. Current production level DBMSs still suffer from insufficient testing due to the limited hand-written test cases. Recently several works proposed to automatically generate many test cases with query transformation, a process of generating an equivalent query pair and testing a DBMS by checking whether the system returns the same result set for the query pair. However, all of the existing works still heavily rely on manual work to provide a transformation which largely confines their exploration of the valid input query space. This thesis introduces duplicate-sensitivity guided transformation synthesis which automatically finds new transformations by first synthesizing many candidates then filtering the nonequivalent ones. Our automated synthesis is achieved by mutating a query while keeping its duplicate sensitivity, which is a necessary condition to preserve query equivalence. After candidate synthesis, we keep the mutant query which is equivalent to the given one by using a query equivalent checker. Furthermore, we have implemented our idea in a tool Eqsql and used it to test the production-level DBMSs. In two months, we detected in total 30 newly confirmed and unique bugs in Oracle MySQL, PingCAP TiDB, and Tencent CynosDB.