Awesome

MACKE - `M`odular `a`nd `C`ompositional Analysis with `K`LEE (and AFL) `E`ngine

MACKE is a wrapper around AFL and KLEE, that decomposes the analyzed programs into several smaller units, analyze these seperately and finally merge all found errors to one interactive report. Please read the MACKE-paper for more details.

Installation guide

Requirements

Python 3.4+
A system able to run LLVM. See official requirements for LLVM
AFL See AFL's Quickstart guide

Step 1: LLVM and KLEE with targeted search

Building KLEE can be a complicated task and there are multiple strategies for it. We suggest the setup described in our Step-by-Step manual. But whatever way you choose, MACKE needs a special search mode, that is not part of the official KLEE code, yet. We aim to merge it into KLEE someday, but till then, you need to use our fork of KLEE.

For our step-by-step manual, this means, that you have to adapt one command. Instead of:

git clone --depth 1 --branch v1.3.0 https://github.com/klee/klee.git

in Step 6, you must use:

git clone --depth 1 https://github.com/tum-i22/klee22.git

In addition to the above, you also need to install LLVM 6.0 if you want the ability to fuzz in phase 1 of Macke.

For our step-by-step manual, this means that you must repeat Step 1 for LLVM 6.0, i.e. replace RELEASE_342 with RELEASE_600 in all links.

Step 2: Building the macke-llvm-opt passes

MACKE performs several modifications on LLVM bitcode level. Doing this inside python requires a lot more effort, than directly writing the operations in C++ - especially if you are forced to use the same, old version of LLVM as KLEE uses. Therefore, we decide to seperate all low level operations into two other repositories - one for LLVM 3.4 for KLEE-related stuff and another one for LLVM 6.0 for AFL-related stuff.

If you choose a different directory structure than suggested in our Step-by-Step manual, please adapt the pathes to match your needs.

git clone --depth 1 https://github.com/tum-i22/macke-opt-llvm 
cd macke-opt-llvm
make LLVM_SRC_PATH=~/build/llvm3.4/ KLEE_BUILDDIR=~/build/klee/Release+Asserts KLEE_INCLUDES=~/build/klee/include/

Now repeat the above step for macke-fuzzer-opt-llvm

git clone --depth 1 https://github.com/tum-i22/macke-fuzzer-opt-llvm 
cd macke-fuzzer-opt-llvm
make LLVM_SRC_PATH=~/build/llvm6.0/ KLEE_BUILDDIR=~/build/klee/Release+Asserts KLEE_INCLUDES=~/build/klee/include/

Step 3: Building MACKE

We are done with the dependencies - now to the main project.

# You might have to change the branch in repository below, depending on the version you want to build
git clone --depth 1 https://github.com/tum-i22/macke
cd macke
make dev

Step 4: Running MACKE

Before you can actually start using MACKE, you have to modify the ./config.ini with your favorite text editor. Please adapt the pathes there to the directories, you have created earlier in this guide.

First switch your virtual environment to Macke

source .venv/bin/activate # Note: just needed once per open shell

If you want to analyze the isolated functions with symbolic execution then run the following:

macke 2beAnalyzed.bc

Otherwise if you want to analyze the isolated functions with fuzzing (AFL) then run the following:

macke --use-fuzzer=1 --fuzz-bc=2beAnalyzedCompiledWithClang6.0.bc 2beAnalyzed.bc

Finally, if you want to analyze the isolated functions with flipper (switch between KLEE and AFL, whenever one of them saturates) then run the following:

macke --flipper=1 --fuzz-bc=2beAnalyzedCompiledWithClang6.0.bc 2beAnalyzed.bc

We wish you happy testing! If you have problems converting existing source code bases to bitcode files, you should have a look at this tool for improving make.

Troubleshooting

Getting around the issue of cgroups

Linux control groups, or Cgroups in short, are a kernel feature that allows user space processes to have limited (and exclusive) access to certain system resources, such as CPU. We leverage cgroups to isolate parallel fuzzing processes so that they don't interfere.

Therefore, before using Macke, you need to create these cgroups partitions. There are two alternative ways to do this.

Alternative 1: Using Macke

Change as root

sudo -s

Load macke (with source command) virtual environment and create cgroups

macke --initialize-cgroups --cgroup-name=<user>:<group>

If the warning about limiting swap memory shows up, then run the following

macke --initialize-cgroups --cgroup-name=<user>:<group> --ignore-swap

and in all subsequent macke commands, also add --ignore-swap

Alternative 2: Manually on command-line:

Run cgroups command. If your CPU has x cores, then run the following command x times replacing the index i with the CPU number, i.e. mackefuzzer_1, mackefuzzer_2... mackefuzzer_x.

cgcreate -s 775 -d 775 -f 775 -a <usergroup> -t <usergroup> -g memory: mackefuzzer_<i>

You almost certainly might need to run this command as root or with sudo.

Then, in all subsequent macke commands, also add --ignore-swap, if your operating system does not allow partitioning swap memory resource.

Author's note

For current issues, suggestions, datasets and gratitude please email me. Big thanks to HuTo and Fabian Kilger for much of the development effort.

Saahil Ognawala