Karine Even-Mendoza, King’s College London.
Compiler fuzzing techniques require a means of generating programs that are free from undefined behaviour (UB) to reliably reveal miscompilation bugs. Existing program generators such as Csmith achieve UB freedom by heavily restricting the form of generated programs. The idiomatic nature of the resulting programs risks limiting the test coverage they can offer and, thus, the compiler bugs they can discover.
In this talk, I will present a new fuzzer, CsmithEdge, which extends Csmith, and via which we investigated the idea of adapting existing fuzzers to be less restrictive concerning UB in the practical setting of C compiler testing. CsmithEdge probabilistically weakens the constraints used to enforce UB freedom thus generated programs are no longer guaranteed to be UB-free. It then employs several off-the-shelf UB detection tools and a novel dynamic analysis to (a) detect UB in generated programs and (b) eliminate UB-freedom constraints for arithmetic operations when Csmith has been too conservative in its use. The resulting UB-free programs are used to test for miscompiled bugs (via differential testing), and the non-UB-free programs are used for crash testing. During our short evaluation, CsmithEdge discovered several previously unknown miscompilation in mature compilers such as GCC, LLVM and MVSC and achieved substantial differences in code coverage on GCC and LLVM compared with regular Csmith.