Home

Awesome

armshaker

armshaker is a processor fuzzer targeting the Armv8-A ISA. It can be used to find hidden instructions in hardware processors or other implementations of the ISA like emulators.

In essence, it works by executing undefined instructions and checking whether the execution generates an undefined instruction exception or not, in the form of a SIGILL signal sent to the process. If no SIGILL is received, the instruction is marked as hidden and logged. The limited size of the three instruction sets in Armv8-A (A64, A32 and T32) makes it feasible to do an exhaustive search over the whole instruction space using this method.

Usage

armshaker has two primary parts: the back-end doing most of the work, and a front-end adding visualizations and multiprocessing support. In most cases, the front-end is the preferred way to run the fuzzer.

The fuzzer must be compiled before use, which can usually be done with a simple

make

in the project directory.

Then an exhaustive search of the default instruction set can be initiated like so:

./armshaker.py -f2

Which will look something like this:

<img src="refs/frontend.gif" alt="drawing" width="600"/>

The particular instruction set that is fuzzed depends on the runtime of the current system. If the fuzzer is compiled with a 32-bit (AArch32) toolchain, it will be able to fuzz A32 or T32 (with the -t option). If it is compiled with a 64-bit toolchain (AArch64), it will be able to fuzz A64, although cross-compiling and running a 32-bit fuzzer from AArch64 is possible.

If a hidden instruction is found, it will be logged in the file data/logX, where X corresponds to the worker ID. Each log entry will be in the following format: <instruction_encoding>,hidden,<generated_signal_number>,..., with register value changes appended if the -p (and optionally -g) option is set.

In case Python 3 is not available, shell_frontend.sh can be used instead for multiprocessing support. Otherwise the fuzzer back-end can be run directly with ./fuzzer <options>.

Building

armshaker is made for Armv8-A-based systems running Linux, although it should be possible to run it on Armv7-A too. Other than Linux, it only requires a C compiler (GCC), the make tool and Python 3 for the front-end.

The simplest way to compile the fuzzer is to run

make

in the project directory. This will use the bundled libopcodes disassembler and should work for most use cases.

Optionally, a local installation of libopcodes can be used instead with

make SHARED_LIBOPCODES=TRUE

Capstone can also be used in addition to libopcodes (note: not instead of). This is mostly a legacy feature, but can be used to compare disassembly results between the two. It can be enabled by adding the USE_CAPSTONE=TRUE option when compiling.

Options

The options available in the front-end are as follows. For more detailed descriptions, see the options for the back-end.

$ ./armshaker.py -h
usage: armshaker.py [-h] [-s INSN] [-e INSN] [-c] [-w NUM] [-p] [-n]
                    [-f LEVEL] [-t] [-z] [-g] [-V] [-c]

fuzzer front-end

optional arguments:
  -h, --help            show this help message and exit
  -s INSN, --start INSN
                        search range start
  -e INSN, --end INSN   search range end
  -d, --discreps        Log disassembler discrepancies
  -w NUM, --workers NUM
                        Number of worker processes
  -p, --ptrace          Use ptrace when testing
  -n, --no-exec         Don't execute instructions, just disassemble them.
  -f LEVEL, --filter LEVEL
                        Filter certain instructions
  -t, --thumb           Use the thumb instruction set (only on AArch32).
  -z, --random          Load the registers with random values, instead of all
                        0s.
  -g, --log-reg-changes
                        For hidden instructions, only log registers that
                        changed value.
  -V, --vector          Set and log vector registers (d0-d31, fpscr) when
                        fuzzing.
  -c, --cond            Set cpsr flags to match instruction condition code.

The back-end has some extra options that can be useful for analysis or targeted fuzzing. Its options are as follows.

$ ./fuzzer -h
Usage: ./fuzzer [option(s)]

General options:
    -h, --help              Print help information.
    -q, --quiet             Don't print the status line.

Search options:
    -s, --start <insn>      Start of instruction search range (in hex).
                            [default: 0x00000000]
    -e, --end <insn>        End of instruction search range, inclusive (in
                            hex). [default: 0xffffffff]
    -i, --single-exec       Execute a single instruction (i.e., set end=start).
    -m, --mask <mask>       Only update instruction bits marked in the supplied
                            mask. Useful for testing different operands on a
                            single instruction. Example: 0xf0000000 -> only
                            increment most significant nibble.

Execution options:
    -n, --no-exec           Calculate the total amount of undefined
                            instructions, without executing them.
    -x, --exec-all          Execute all instructions (regardless of the
                            disassembly result).
    -f, --filter <level>    Filter away (skip) certain instructions that would
                            otherwise be executed and might generate false
                            positives. Supports the following levels, where
                            each level includes the numerically lower ones:
                                1: Inaccurate disassemblies, primarily caused
                                   by SBO/SBZ bits.
                                2: Hidden instructions induced by Linux
                                   as a result of bugs or backwards
                                   compatibility measures.
    -p, --ptrace            Execute instructions on a separate process using
                            ptrace. This will generally make execution slower,
                            but lowers the chance of the fuzzer crashing in
                            case hidden instructions with certain side-effects
                            are found. It also enables some additional options.
    -c, --cond              On AArch32: Set the condition flags in the CPSR to
                            match the condition code in the instruction
                            encoding. This ensures that undefined instructions
                            with a normally non-matching condition code won't
                            be skipped, as is the case in some ISA
                            implementations.

Logging options:
    -l, --log-suffix        Add a suffix to the log and status file.
    -d, --discreps          Log disassembler discrepancies.

Ptrace options (only available with -p option):
    -t, --thumb             Use the thumb instruction set (only available on
                            AArch32). Note: 16-bit thumb instructions have the
                            format XXXX0000. So to test e.g. instruction 46c0,
                            use 46c00000.
    -r, --print-regs        Print register values before/after instruction
                            execution.
    -z, --random            Load the registers with random values, instead of
                            all 0s. Note that the random values are generated
                            at startup, and remain constant throughout the
                            session.
    -g, --log-reg-changes   For hidden instructions, only log registers that
                            changed value.
    -V, --vector            Set and log vector registers (d0-d31, fpscr) when
                            fuzzing.

More Usage Examples

Fuzz the 16-bit part of T32 (Thumb):

$ ./fuzzer -e e7ff0000 -pt -f2

Fuzz the 32-bit part of T32 (using multiple processes):

$ ./armshaker.py -s e8000000 -pt -f2

Check if your (64-bit) Linux kernel induces undocumented 32-bit T32 SETEND instructions:

$ sudo sh -c 'echo 1 > /proc/sys/abi/setend'  # Enable SETEND emulation
$ ./fuzzer -s e800b650 -m ffff0008 -ptg -f1

Check if your QEMU release has hidden VMUL instructions (in A32/T32):

$ ./fuzzer -s f3200d10 -m 004ff0ef -pVzg -f2

Check if your QEMU release has hidden VQDMULL instructions (in A32/T32):

$ ./fuzzer -s f3900d00 -m 007ff0af -pVzg -f2

Execute a defined instruction and see its effects (A64 example):

$ tools/as.sh 'mov x0, #0x1337'
d28266e0
$ ./fuzzer -s d28266e0 -i -pxr
insn: 0xd28266e0, checked: 0, skipped: 0, filtered: 0, hidden: 0, ips: 27676236
insn: d28266e0
x0:     0000000000000000        0000000000001337
x1:     0000000000000000        0000000000000000
x2:     0000000000000000        0000000000000000
...
x29:    0000000000000000        0000000000000000
x30:    0000000000000000        0000000000000000
sp:     0000000000000000        0000000000000000
pc:     000000555ba769f0        000000555ba769ec
pstate: 0000000000000000        0000000000000000
signal: 0

Troubleshooting

The fuzzer detects millions of hidden instructions in A32. Is something wrong?

It's possible that your particular processor executes undefined instructions with an unmatched condition code as NOPs, instead of generating an exception. This is an implementation defined behavior, as documented in section G1.16.1 of the DDI0487E Armv8 Architecture Reference Manual.

This behavior can in some sense be bypassed by making sure the condition code is always matched, which is what the -c option does. See if adding that option reduces the number of reported hidden instructions.

References

armshaker was inspired by Christopher Domas's sandsifter project and implemented as part of my master's thesis in computer science, where I used it to fuzz a variety of Armv8-A-based systems. No hidden instructions that could be attributed to hardware were found, but the fuzzing did reveal bugs in the QEMU emulator and the Linux kernel. See the refs directory for more information.

Citing

The main findings of my thesis were published as an article presented at the HASP 2020 workshop, under the title Uncovering Hidden Instructions in Armv8-A Implementations (preprint version).

If you would like to cite this work, you can use the following BibTex entry:

@inproceedings{10.1145/3458903.3458906,
  author = {Strupe, Fredrik and Kumar, Rakesh},
  title = {Uncovering Hidden Instructions in Armv8-A Implementations},
  year = {2021},
  isbn = {9781450388986},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3458903.3458906},
  doi = {10.1145/3458903.3458906},
  booktitle = {Hardware and Architectural Support for Security and Privacy},
  articleno = {3},
  numpages = {9},
  keywords = {undocumented instructions, Armv8-A, hidden instructions, hardware security},
  location = {Virtual, Greece},
  series = {HASP '20}
}