Awesome
lxfuzz
lxfuzz is a grey-box kernel fuzzer used for linux. it is scalable because of qemu being used to emulate in a way to be able to freely choose the number of instances and their memory.
setup and run
first, install build dependencies (example debian-based):
wget git make gcc g++ flex bc bison pkg-config ninja-build libssl-dev libglib2.0-dev bzip2 libpixman-1-dev libelf-dev libncurses5-dev
of course, you will need a properly built linux kernel. following options should be enabled
CONFIG_USER_NS=y
CONFIG_NET_DEV_REFCNT_TRACKER=y
CONFIG_NET_NS_REFCNT_TRACKER=y
CONFIG_KASAN=y
CONFIG_PANIC_ON_OOPS=y
CONFIG_BUG_ON_DATA_CORRUPTION=y
CONFIG_KCOV=y # make sure /sys/kernel/debug/kcov is rw for user
CONFIG_KCOV_INSTRUMENT_ALL=y
CONFIG_9P_FS=y
CONFIG_9P_FS_POSIX_ACL=y
CONFIG_9P_FS_SECURITY=y
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NET_9P=y
CONFIG_NET_9P_DEBUG=y
CONFIG_NET_9P_VIRTIO=y
CONFIG_DEBUG_INFO=y
# CONFIG_RANDOMIZE_BASE is not set
enabling extra options that add more code to be fuzzed is always a good idea
now, to build the fuzzer simply run
make all
this will build a custom qemu emulator (x86-64) plus the fuzzer, a reproducer and manager programs for both of those.
before running the manager, you have to configure how QEMU should run your kernel. you can do that by editing the cmdline.cfg
file (which already contains an example configuration)
you're completely free in choosing how the kernel should be running. however make sure to have qemu exit on a kernel panic or similar. be careful to use the modified qemu emulator. (located in ./tools/qemu-7.1.0/build/
)
if everything is set up you can start the fuzzing manager
./fuzz_manager -n <instances> [--timeout <inactive log timeout>] [--daemon] [--userns]
required arguments:
with -n
, you can choose as many qemu instances as your hardware can take.
optional arguments:
--timeout
specifies in seconds, how long no log activity should be ignored, until the fuzz_manager
checks for hangs or crashes. (default 60s)
use --daemon
to run the fuzzer as a daemon in the background.
--userns
tells the fuzzer to make use of user namespaces.
coverage
lxfuzz uses kcov for coverage information gaining. reached addresses in the kernel are saved in ./coverage/kcov.txt
, which is accessible by the guests and host. it is essential for lxfuzz
to function properly, so make sure the path exists. this information can also be processed by syzkallers's syz-cover
.
logs and crashes
all fuzzing logs are saved in ./kernel/data/
. each instance got an own directory in which each core/thread got an own log file.
WARNING: the log folders and files grow extremely large after some time. make sure to keep track of them and keep removing old log data (i.e. by a shell script)
if the manager encounters a crash, the whole log directory of the corresponding instance is copied and saved. the active logfiles get cleared whenever a crash or hang occurs
to reproduce crashes, copy the folder containing the crashes to the default startup working directory of the machine alongside with the reproducer
binary. make sure the folder is named crash/
. running repro_manager
will try to reproduce the crash and will notify if successful. you can try to reduce the log data in crash/
while still reproducing successfully, until you get to a point where you're able to understand the crash and create a POC.