Home

Awesome

Interesting Fuzz-mg

Coverage-based Greybox Fuzzing as Markov Chain(CCS 16)

T-Fuzz: fuzzing by program transformation(Oakland 18)

T-Fuzz Design

CollAFL: Path Sensitive Fuzzing(Oakland 18)

该paper主要对AFL有两个改进:

我个人认为,该论文的主要贡献是提供了一个机制来解决路径的hash collision问题,使得coverage判断更加准确。

AFL Coverage Measurements

AFL使用bitmap(默认64KB)来跟踪edge coverage。没一个字节都对应特定edge的hit count。AFL通过对每个basic block进行插桩,为每个basic block都随机分配一个id,当执行每条路径时,对该路径上的每个basic block都进行如下操作:

cur_location= <COMPILE_TIME_RANDOM>;
shared_mem[cur_location ^ prev_location]++;
prev_location = cur_location >> 1;

其中上面的prev_location右移一位主要是为了区分路径A->B和B->A。由于每个basic block的id是随机分配的,所以这种hash方法很容易产生collision,特别当程序比较大的时候,collision rate也越大。

CollAFL's Solution to Hash Collision

CollAFL通过三种方式来解决hash collision:

  1. 公式1 通过贪心算法,为每个basic block分配x和y的值,保证每条edge计算的hash值都是不同的。
  2. 如果每个basic block只有一个前继basic block,即只有一条边到达该basic block,所以只需要将该basic block的id来表示该edge即可。
  3. 如果前面两种方法无法解决,则动态的时候为每条边分配不同的id。

VUzzer: Application-aware Evolutionary Fuzzing(NDSS 17)

VUzzer是公认的比较好的类AFL fuzzer。它主要利用Data-flow features和Control-flow features来辅助fuzzer变异和进行seed的选择。

Data-flow features

利用dynamic taint analysis 来推断input的结构和类型,以及某段数据在input的偏移。比如,它通过对每个cmp指令进行插桩来判断input的哪些字节与输入有关,并且知道与它比较的另外一个值。同时,VUzzer也可以对lea指令进行插桩,从而检测index操作是不是与input某些bytes有关。

Control-flow features

Control-flow features可以让VUzzer推断出执行路径的重要性。比如,某些执行路径最后到达了error-handling blocks。VUzzer就通过静态的方法识别出了一下error-handling code。同时,VUzzer通过对每个basic block赋予特定的权重,来促使fuzzer走到更深的路径中去。

Angora: Efficient Fuzzing by Principled Search(Oakland 18)

This paper's contributions:

Designing New Operating Primitives to Improve Fuzzing Performance(CCS 17)

AFL Overview

afl-overview

QSYM: A Practical Concolic Execution Engine Tailored for Hybrid Fuzzing(Usenix 18)

该paper是Usenix 18的Distinguished Paper,其主要针对了当前的concolic execution的三个方面进行了优化: Slow Symbolic Emulation, Ineffective Snapshot and Slow and Inflexible Sound Analysis. 从而使得concolic execution更好的适应fuzzing场景。

Send Hardest Problems My Way: Probabilistic Path Prioritization for Hybrid Fuzzing(NDSS 19)

SAVIOR: Towards Bug-Driven Hybrid Testing(Oakland 20)

PANGOLIN: Incremental Hybrid Fuzzing with Polyhedral Path Abstraction(Oakland 20)

To be incremental, they propose “polyhedral path abstraction”, which preserves the exploration state in the concolic execution stage and allows more effective mutation and constraint solving over existing techniques.

Symbolic execution with SymCC: Don't interpret, compile!(Usenix Security 20)

Motivation: Performance Bottlenecks

Slow Symbolic Emulation

现在主流的concolic executors做符号执行的时候是针对IR中间语言做的(比如KLEE的LLVM IR和angr的VEX IR),对中间语言模拟执行。其采用IR的原因是实现起来比较简单。由于Intel 64位指令集包含1795条指令,所以针对每条指令总结出来符号的语义对于人工来说是一个非常大的工作量,而IR的指令较少(LLVM IR有62条指令),符号化这些指令相对比较简单。

然而使用IR则引发了额外的overhead。首先,从机器指令到IR的转换本身就有overhead。由于amd64是CISC(complex instruction set computer),而IR是RISC(reduced instruction set computer),一般一条amd64的指令需要转换成多条IR指令,拿angr为例,如果将amd64指令转为VEX IR,则平均增加的指令数是4.69倍。其次,采用IR导致basic block level taint。因为由于效率的原因,从native instructions到IR的转换一般是以basic block为单位的,这样就导致无法将单个的native instruction转换成IR,所以也就只能做到哪些basic block需要符号化,而不是具体的某条指令需要符号化。这样做导致的后果就是如果某个basic block中只有一条指令和输入有关需要符号化,则整个basic block都需要符号模拟,这样就会造成很高的overhead。如果没有IR的话就可以做到指令级别的taint,就能够清楚的判断哪些指令需要符号模拟,哪些指令只需native execution,减少了不必要的符号模拟。实验表明,在一个basic block中,只有30%的指令需要符号模拟。

Ineffective Snapshot

snapshot是concolic execution常用的一个技术,它能够保存某条分支前的状态S,当该分支执行完或者"stuck"时,可以从该状态S直接执行另外一个分支,避免了重新执行的overhead。然而snapshot本身就有一些缺点:snapshot需要保存一些外部的状态(文件系统,内存管理系统),则此时需要对影响外部状态的系统调用进行处理,一般有两个方法: full system concolic execution and External environment modeling。这两个方法都有一些缺陷:第一个方法是由于外部环境比较复杂,实现起来比较难,overhead较高;第二个则是model的system call较少,并且有些system call建模的不够完全。另外由于fuzzing的输入一般不会共享同一个分支,所以snapshot可能对于fuzzing这个场景也不是很好,所以该paper就没有采用snapshot的机制,对于每个输入都会重新执行,对于系统调用,则具体执行。

Slow and Inflexible Sound Analysis

现在的concolic execution是将某条路径上的所有constraints都满足,从而求解出具体的input。然而复杂的constraints可能会导致输入求解不出。所以该paper的一个解决方法就是只求解出部分constraints。

FAIRFUZZ: A Targeted Mutation Strategy for Increasing Greybox Fuzz Testing Coverage(ASE 18)

FairFuzz focus on branch coverage, it works in two main steps.

First, it identifies the program branches that are rarely hit by previously-generated inputs. It calls such branches rare branches. These rare branches guard under-explored functionalities of the program. By generating more random inputs hitting these rare branches, FairFuzz greatly increases the coverage of the parts of the code guarded by them.

Second, FairFuzz uses a novel lightweight mutation technique to increase the probability of hitting these rare branches. The mutation strategy is based on the observation that certain parts of an input already hitting a rare branch are crucial to satisfy the conditions necessary to hit that branch. Therefore, to generate more inputs hitting the rare branch via mutation, the parts of the input that are crucial for hitting the branch should not be mutated.

Full-speed Fuzzing: Reducing Fuzzing Overhead through Coverage-guided Tracing(Oakland 19)

ProFuzzer: On-the-fly Input Type Probing for Better Zero-day Vulnerability Discovery(Oakland 19)

NEUZZ: Efficient Fuzzing with Neural Program Smoothing(Oakland 19)

REDQUEEN: Fuzzing with Input-to-State Correspondence(NDSS 19)

NAUTILUS: Fishing for Deep Bugs with Grammars(NDSS 19)

Send Hardest Problems My Way: Probabilistic Path Prioritization for Hybrid Fuzzing(NDSS 19)

EnFuzz: Ensemble Fuzzing with Seed Synchronization among Diverse Fuzzers(Usenix Security 19)

MOPT: Optimize Mutation Scheduling for Fuzzers(Usenix Security 19)

GRIMOIRE: Synthesizing Structure while Fuzzing(Usenix Security 19)

Ptrix: Efficient Hardware-Assisted Fuzzing for COTS Binary(AsiaCCS 19)

Matryoshka: Fuzzing Deeply Nested Branches(CCS 19)

GREYONE: Data Flow Sensitive Fuzzing(Usenix Security 20)

The key insight in this paper is to add data flow (or more specifically, taint information) to help guide the fuzzing. It adds dynamic taint tracking, and propose several new seed selection policies and mutation strategies.

IJON: Exploring Deep State Spaces via Fuzzing(Oakland 20)

In addition to considering the input that reaches new code coverage as seed, it also considers the state spaces(such as the value of a specific variable). Nice try!

Not All Coverage Measurements Are Equal: Fuzzing by Coverage Accounting for Input Prioritization(NDSS 20)

To address the limitation caused by equal coverage, they propose coverage accounting, a novel approach that evaluates coverage by security impacts. Coverage accounting attributes edges by three metrics based on three different levels: function, loop and basic block. Based on the proposed metrics, they design a new scheme to prioritize fuzzing inputs and develop TortoiseFuzz, a greybox fuzzer for finding memory corruption vulnerabilities. They evaluated TortoiseFuzz on 30 real-world applications and compared it with 6 state-of-the-art greybox and hybrid fuzzers: AFL, AFLFast, FairFuzz, MOPT, QSYM, and Angora. Statistically, TortoiseFuzz found more vulnerabilities than 5 out of 6 fuzzers (AFL, AFLFast, FairFuzz, MOPT, and Angora), and it had a comparable result to QSYM yet only consumed around 2% of QSYM’s memory usage on average. They also compared coverage accounting metrics with two other metrics, AFL-Sensitive and LEOPARD, and TortoiseFuzz performed significantly better than both metrics in finding vulnerabilities. Furthermore, they applied the coverage accounting metrics to QSYM and noticed that coverage accounting helps increase the number of discovered vulnerabilities by 28.6% on average. TortoiseFuzz found 20 zero-day vulnerabilities with 15 confirmed with CVE identifications

Montage: A Neural Network Language Model-Guided JavaScript Engine Fuzzer (Usenix Security 20)

This paper proposes neural network language models (NNLMs) guided fuzzer to find bugs in JavaScript engines. The key technique is to reuse existing JavaScript test cases in regression tests, and use NNLMs over ASTs of the JS code to generate better fuzzing inputs.

Fuzzing JavaScript Engines with Aspect-preserving Mutation (Oakland 20)

Hybrid Fuzzing

Driller: Augmenting Fuzzing Through Selective Symbolic Execution(NDSS 16)

QSYM: A Practical Concolic Execution Engine Tailored for Hybrid Fuzzing(Usenix 18)

该paper是Usenix 18的Distinguished Paper,其主要针对了当前的concolic execution的三个方面进行了优化: Slow Symbolic Emulation, Ineffective Snapshot and Slow and Inflexible Sound Analysis. 从而使得concolic execution更好的适应fuzzing场景。

我们都知道,fuzzing对于一些比较宽松的限制(比如x>0)能够很容易的通过变异产生一些输入达到该条件;而symbolic execution非常擅长求解一下magic value(比如x == deadleaf)。这是一篇比较经典的将concolic execution和fuzzing结合在一起的文章,该文章的主要思想就是先用AFL等Fuzzer根据seed进行变异,来测试程序。当产生的输入一直走某些路径,并没有探测到新的路径时,此时就"stuck"了。这时,就是用concolic execution来产生输入,保证该输入能走到一些新的分支。从而利用concolic execution来辅助fuzz。

Directed Fuzzing

Directed Greybox Fuzzing(CCS 17)

Input: Seed Input S

repeat
    s = CHOOSENEXT(S)
    p = ASSIGNENERGY(s)    //This paper focus
    for i from 1 to p do
        s' = MUTATE_INPUT(s)
        if t' crashes then
            add s' to Sx
        else if ISINTERESTING(s') then
            add s' to S
        end if
    end for
until timeout reached or abort-signal

Output: Crashing Inputs Sx

类AFL的fuzzing一般步骤如上所示,该paper主要关注于ASSIGNENERGY(s)这一操作,他们通过对不同的seed s赋予不同的energy,即如果一个seed s'产生的trace距离目标基本块targetB较近,则其energy(p)就较大,基于种子s'进行的变异操作就会变多。所以该paper主要有两个contributions: 设计一套算法计算seed s'产生的trace与targetB的距离;通过模拟退火算法来为每个seed s分配energy。

Hawkeye: Towards a Desired Directed Grey-box Fuzzer(CCS 18)

Desired Properties of Directed Fuzzing

AFLGo's Solution

Suggestions to improve DGFs:

Hawkeye's Design

overview

During fuzzing, the fuzzer selects a seed from a priority seed queue. The fuzzer applies a power scheduling against the seed with the goal of giving those seeds that are considered to be "closer" to the target sites more mutation chances, i.e, energy. Specifically, this is achieved through a power function, which is a combination of the covered function similarity and the basic block trace distance. For each newly generated test seed during mutation, after capturing its execution trace, the fuzzer will calculate the covered function similarity and the basic block trace distance based on the utilities. For each input execution trace, its basic block trace distance is calculated as the accumulated basic block level distances divided by the total number of executed basic blocks; and its covered function similarity is calculated based on the overlapping of current executed functions and the target function trace closure, as well as the function level distance.

After the energy is determined, the fuzzer adaptively allocates mutation budgets on two different categories of mutations according to mutators' granularities on the seed(coarse-grained mutations and fine-grained mutations). Afterward, the fuzzer evaluates the newly generated seeds to prioritize those that have more energy or that have reached the target functions.

FuzzGuard: Filtering out Unreachable Inputs in Directed Grey-box Fuzzing through Deep Learning

This paper uses deep learning algorithms to filter out unreachable inputs while maintaining acceptable performance.

Fuzzing Machine Learning Model

TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing(18)

Coverage-Guided Fuzzing for Deep Neural Networks(18)

Kernel Fuzzing

RAZZER: Finding Kernel Race Bugs through Fuzzing(Oakland 19)

kAFL: Hardware-Assisted Feedback Fuzzing for OS Kernels(Usenix Security 17)

Fuzzing File Systems via Two-Dimensional Input Space Exploration(Oakland 19)

This paper proposes an evolutionary feedback-driven fuzzer, Janus, that explores both file system images and file system operations (system calls) to find memory safety bugs. Janus analyzes the data structures in different file systems and only mutates metadata blocks of file system image. This reduces the search space. According to the runtime status in the file system during fuzzing, Janus selectively mutates system calls. Besides, Janus leverages a Lib-OS based executor to start fuzzing from a fresh-state, which avoids the influences of accumulated internal states and eases bug reproducing,

Finding Semantic Bugs in File Systems with an Extensible Fuzzing Framework (SOSP 19)

This paper modulizes the file system fuzzing into Hydra, and bug checkers. Hydra is responsible for input generation and path exploration, while developers can write their own bug checkers of their own interests. This work also integrates Janus, however, it focuses more on semantic bugs, which are not studied in Janus.

PeriScope: An Effective Probing and Fuzzing Framework for the Hardware-OS Boundary(NDSS 19)

KRACE: Data Race Fuzzing for Kernel File Systems(Oakland 20)

It proposes a new coverage tracking metric, alias coverage, specially designed to capture the exploration progress in the concurrency dimension.

HFL: Hybrid Fuzzing on the Linux Kernel(NDSS 20)

To this end, this paper proposes HFL, which not only combines fuzzing with symbolic execution for hybrid fuzzing but also addresses kernel-specific fuzzing challenges via three distinct features: 1) converting indirect control transfers to direct transfers, 2) inferring system call sequence to build a consistent system state, and 3) identifying nested arguments types of system calls. As a result, HFL found 24 previously unknown vulnerabilities in recent Linux kernels. Additionally, HFL achieves 15% and 26% higher code coverage than Moonshine and Syzkaller, respectively, and over kAFL/S2E/TriforceAFL, achieving even four times better coverage, using the same amount of resources (CPU, time, etc.). Regarding vulnerability discovery performance, HFL found 13 known vulnerabilities more than three times faster than Syzkaller.

Agamotto: Accelerating Kernel Driver Fuzzing with Lightweight Virtual Machine Checkpoints(Usenix Security 20)

USBFuzz: A Framework for Fuzzing USB Drivers by Device Emulation(Usenix Security 20)

Hypervisor Fuzzing

HYPER-CUBE: High-Dimensional Hypervisor Fuzzing(NDSS 2020)

Anti-Fuzzing

FUZZIFICATION: Anti-Fuzzing Techniques(Usenix 19)

ANTIFUZZ: Impeding Fuzzing Audits of Binary Executables(Usenix 19)

IoT Fuzzing

IOTFUZZER: Discovering Memory Corruptions in IoT Through App-based Fuzzing(NDSS 18)

FIRM-AFL: High-Throughput Greybox Fuzzing of IoT Firmware via Augmented Process Emulation(Usenix Security 19)

P2IM: Scalable and Hardware-independent Firmware Testing via Automatic Peripheral Interface Modeling(Usenix Security 20)

Evaluate Fuzzing

Evaluating Fuzz Testing(CCS 18)

They found that:

Smart Contract Fuzzing

Learning to Fuzz from Symbolic Execution with Application to Smart Contracts(CCS 19)

Web Fuzzing

Enemy of the State: A State-Aware Black-Box Web Vulnerability Scanner (USENIX Security 12)

Complex web applications have many states. For example, a user has to login first to view certain content. Otherwise, the view page cannot be fuzzed. This paper proposes a state-aware black-box fuzzing with algorithms to infer and distinguish application states.

JS Fuzzing

Fuzzing with Code Fragments (USENIX Security 12)

Fuzzing JavaScript Engines with Aspect-preserving Mutation (SP 19)

CodeAlchemist: Semantics-Aware Code Generation to Find Vulnerabilities in JavaScript Engines (NDSS 21)

Montage: A Neural Network Language Model-Guided JavaScript Engine Fuzzer (USENIX Security 20)

Favocado: Fuzzing the Binding Code of JavaScript Engines Using Semantically Correct Test Cases (NDSS 21)

Automated Conformance Testing for JavaScript Engines via Deep Compiler Fuzzing (PLDI 21)

Other Resources

Fuzzilli

Fuzz javascrpt language by mutating on Fuzzil(an IR).

FuzzOS

gustave

Contributing

Welcome to contribute! Please see instructions here.