Linux Kernel System Call Fuzzing

Mgr. Arvind Rao

Diplomová práce

Anotace:

Exisiting Linux kernel system call fuzzing techniques involve passing of completely random or semi-random(using templates) values as parameters to system calls to find bugs and corner cases. State of the art fuzzers successfully utilize evolutionary coverage guided techniques to improve the quality of the generated random values. Such techniques however do not detect deep bugs in the kernel. System calls, like any other API, is highly interdependent. Fuzzing such interdependent relations between the system calls require the creation of a valid context. This valid context is created when a sequence of system calls is made. The thesis builds upon the work of an existing coverage guided Linux System call fuzzer, TriforceAFL\cite{tafl}, by modifying it to support value passing in a sequence of system calls. A novel method utilizing deep learning and assisted regular expression parsing to generate high quality seeds for the modified fuzzer is also presented. Compared to the original TriforceAFL fuzzer, the modified fuzzer is able to cover more code paths and find more bugs in the kernel. The developed framework was also utilized to fuzz a proprietary DRDO Linux Kernel(DLK). The fuzzing framework was able to discover a previously undetected buffer overrun bug in the implementation.

Abstract:

Exisiting Linux kernel system call fuzzing techniques involve passing of completely random or semi-random(using templates) values as parameters to system calls to find bugs and corner cases. State of the art fuzzers successfully utilize evolutionary coverage guided techniques to improve the quality of the generated random values. Such techniques however do not detect deep bugs in the kernel. System calls, like any other API, is highly interdependent. Fuzzing such interdependent relations between the system calls require the creation of a valid context. This valid context is created when a sequence of system calls is made. The thesis builds upon the work of an existing coverage guided Linux System call fuzzer, TriforceAFL\cite{tafl}, by modifying it to support value passing in a sequence of system calls. A novel method utilizing deep learning and assisted regular expression parsing to generate high quality seeds for the modified fuzzer is also presented. Compared to the original TriforceAFL fuzzer, the modified fuzzer is able to cover more code paths and find more bugs in the kernel. The developed framework was also utilized to fuzz a proprietary DRDO Linux Kernel(DLK). The fuzzing framework was able to discover a previously undetected buffer overrun bug in the implementation.

Keywords

security, software testing, fuzzing, kernel fuzzing, coverage-guided fuzzing, system call fuzzing

Jazyk práce: angličtina

Datum vytvoření / odevzdání či podání práce: 13. 12. 2018

Obhajoba závěrečné práce

• Obhajoba proběhla 31. 1. 2019
• Vedoucí: RNDr. Petr Švenda, Ph.D.
• Oponent: Ing. Mgr. et Mgr. Zdeněk Říha, Ph.D.

Masarykova univerzita

Fakulta informatiky

Magisterský studijní program / obor:
Informatika / Bezpečnost informačních technologií (angl.)