Home

Awesome

FwAnalyzer (Firmware Analyzer)

CircleCI

FwAnalyzer is a tool to analyze (ext2/3/4), FAT/VFat, SquashFS, UBIFS filesystem images, cpio archives, and directory content using a set of configurable rules. FwAnalyzer relies on e2tools for ext filesystems, mtools for FAT filesystems, squashfs-tools for SquashFS filesystems, and ubi_reader for UBIFS filesystems. cpio for cpio archives. SELinux/Capability support for ext2/3/4 images requires a patched version of e2tools. SELinux/Capability support for SquashFS images requires a patched version of squashfs-tools.

fwanalyzer

Overview

The main idea of FwAnalyzer is to provide a tool for rapid analysis of filesystem images as part of a firmware security Q&A check suite. FwAnalyzer takes a configuration file that defines various rules for files and directories and runs the configured checks against a given filesystem image. The output of FwAnalyzer is a report, which contains the list of files that violate any of the rules specified in the configuration. The report further contains meta information about the filesystem image and, if configured, information extracted from files within the analyzed filesystem. The report is formatted using JSON so it can be easily integrated as a step in a larger analysis.

Example report:

{
    "fs_type": "extfs",
    "image_digest": "9d5fd9acc98421b46976f283175cc438cf549bb0607a1bca6e881d3e7f323794",
    "image_name": "test/test.img",
    "current_file_tree_path": "test/oldtree.json.new",
    "old_file_tree_path": "test/oldtree.json",
    "data": {
        "Version": "1.2.3",
        "date1 file": "Mon Oct  1 16:13:05 EDT 2018\n"
    },
    "informational": {
        "/bin": [
                "CheckFileTree: new file: 40755 1001:1001 1024 0 SeLinux label: -"
        ],
    },
    "offenders": {
        "/bin/elf_arm32": [
                "script(check_file_elf_stripped.sh) returned=elf_arm32 is not stripped"
        ],
        "/file1": [
                "File not allowed"
        ],
        "/file2": [
                "File is WorldWriteable, not allowed",
                "File Uid not allowed, Uid = 123"
        ],
    }
}

Building and Development

Follow the steps described in Building to install all requirements and build FwAnalyzer.

Using FwAnalyzer

Command line options

Example:

fwanalyzer -cfg system_fwa.toml -in system.img -out system_check_output.json

Example for using custom scripts stored in the scripts/ directory:

PATH=$PATH:./scripts fwanalyzer -cfg system_fwa.toml -in system.img -out system_check_output.json

The devices/ folder contains helper scripts for unpacking and dealing with specific device types and firmware package formats such as Android. It also includes general configuration files that can be included in target specific FwAnalyzer configurations.

check.py in the devices/ folder provides a universal script to effectively use FwAnalyzer, see devices/Readme.md for details. This likely is how most people will invoke FwAnalyzer.

The scripts/ folder contains helper scripts that can be called from FwAnalyzer for file content analysis and data extraction. Most interesting should be our checksec wrapper check_sec.sh, see the Checksec Wrapper Readme.

Config Options

Global Config

The global config is used to define some general parameters.

The FsType (filesystem type) field selects the backend that is used to access the files in the image. The supported options for FsType are:

The FsTypeOptions allow tuning of the FsType driver.

The DigestImage option will generate a SHA-256 digest of the filesystem image that was analyzed, the digest will be included in the output.

Example:

[GlobalConfig]
FsType        = "extfs"
FsTypeOptions = "selinux"
DigestImage   = true

Example Output:

"fs_type": "extfs",
"image_digest": "9d5fd9acc98421b46976f283175cc438cf549bb0607a1bca6e881d3e7f323794",
"image_name": "test/test.img",

Include

The Include statement is used to include other FwAnalyzer configuration files into the configuration containing the statement. The include statement can appear in any part of the configuration.

The -cfgpath parameter sets the search path for include files.

Example:

[Include."fw_base.toml"]

Global File Checks

The GlobalFileChecks are more general checks that are applied to the entire filesystem.

Example:

[GlobalFileChecks]
Suid          = true
SuidAllowedList = ["/bin/sudo"]
SELinuxLabel  = false
WorldWrite    = true
Uids          = [0,1001,1002]
Gids          = [0,1001,1002]
BadFiles      = ["/file99", "/file1", "*.h"]

Example Output:

"offenders": {
  "/bin/su": [ "File is SUID, not allowed" ],
  "/file1":  [ "File Uid not allowed, Uid = 123" ],
  "/world":  [ "File is WorldWriteable, not allowed" ],
}

Link Handling

With links we refer to soft links. Links can point to files on a different filesystem, therefore, we handle them in a special way. Link handling requires a patched version of e2tools:

FileStatCheck will handle links like you would expect it. However if AllowEmpty is false and the file is a link then the check fails.

All other checks and dataextract will fail if the file is a link. Those checks need to be pointed to the actual file (the file the link points to).

File Stat Check

The FileStatCheck can be used to model the metadata for a specific file or directory. Any variation of the configuration will be reported as an offender.

Example:

[FileStatCheck."/etc/passwd"]
AllowEmpty = false
Uid        = 0
Gid        = 0
Mode       = "0644"
Desc       = "this need to be this way"

Example Output:

"offenders": {
  "/file2": [ "File State Check failed: size: 0 AllowEmpyt=false : this needs to be this way" ],
}

File Path Owner Check

The FilePathOwner check can be used to model the file/directory ownership for a entire tree of the filesystem. The check fails if any file or directory with in the given directory is not owned by the specified Uid and Gid (type: int).

Example:

[FilePathOwner."/bin"]
Uid = 0
Gid = 0

Example Output:

"offenders": {
  "/dir1/file3": [ "FilePathOwner Uid not allowed, Uid = 1002 should be = 0",
                   "FilePathOwner Gid not allowed, Gid = 1002 should be = 0" ],
}

File Content Check

The FileContent check allows to inspect the content of files. The content of a file can be check using four different methods. The file content check can be run in non enforcement mode by setting InformationalOnly to true (default is false). InformationalOnly checks will produce informational element in place of an offender.

Example: Regular Expression on entire file body

Example:

[FileContent."RegExTest1"]
RegEx = ".*Ver=1337.*"
Match = true
File  = "/etc/version"

Example: SHA-256 digest calculated over the file body

Example:

[FileContent."DigestTest1"]
Digest = "8b15095ed1af38d5e383af1c4eadc5ae73cab03964142eb54cb0477ccd6a8dd4"
File   = "/ver"

Example Output:

"offenders": {
  "/ver": [ "Digest (sha256) did not match found = 44c77e41961f354f515e4081b12619fdb15829660acaa5d7438c66fc3d326df3 should be = 8b15095ed1af38d5e383af1c4eadc5ae73cab03964142eb54cb0477ccd6a8dd4." ],
}

Example: Run an external script passing the filename to the script

The file is extracted into a temp directory with a temp name before the script is executed. The check produces an offender if the script produced output on stdout or stderr.

If the -- is present it indicates that the next argument is from the ScriptOptions[1..N]. The script is run with the following arguments:

<tmp filename> <original filename (fullpath)> <uid> <gid> <mode in octal> <selinux label or "-" for no label> [--] [script argument 1] ... [script argument N]

Example:

[FileContent."ScriptTest1"]
Script = "check_file_x8664.sh"
File   = "/bin"

Example Output:

"offenders": {
  "/bin/elf_arm32": [ "script(check_file_x8664.sh) returned=elf_arm32 not a x86-64 elf file" ],
}

Json Field Compare

Example:

[FileContent."System_Arch"]
Json = "System.Arch:arm64"
File   = "/system.json"
Desc = "arch test"

Example Input:

{
  "System": {
    "Version": 7,
    "Arch": "arm32",
    "Info": "customized"
  }
}

Example Output:

"offenders": {
  "/system.json": [ "Json field System.Arch = arm32 did not match = arm64, System.Arch, arch test" ],
}

File Compare Check

The FileCmp (File Compare) check is a mechanism to compare a file from a previous run with the file from the current run. The main idea behind this check is to provide more insights into file changes, since it allows comparing two versions of a file rather than comparing only a digest.

This works by saving the file as the OldFilePath (if it does not exist) and skipping the check at the first run. In consecutive runs the current file and the saved old file will be copied to a temp directory. The script will be executed passing the original filename, the path to the old file and the path to the current file as arguments. If the script prints output the check will be marked as failed.

Script runs as:

script.sh <OrigFilename> <oldFile> <newFile> [--] [argument 1] .. [argument N]

Example:

[FileCmp."test.txt"]
File = "/test.txt"
Script = "diff.sh"
OldFilePath = "test.txt"
InformationalOnly = true

File Tree Check

The FileTree check generates a full filesystem tree (a list of every file and directory) and compares it with a previously saved file tree. The check will produce an informational output listing new files, deleted files, and modified files.

CheckPath (string array) specifies the paths that should be included in the check. If CheckPath is not set it will behave like it was set to ["/"] and will include the entire filesystem. If CheckPath was set to [] it will generate the file tree but will not check any files.

OldFileTreePath specifies the filename to read the old filetree from, if a new filetree is generated (e.g. because the old filetree does not exist yet) the newly generated filetree file is OldFileTreePath with ".new" appeneded to it.

The OldFileTreePath is relative to the configuration file. This means for '-cfg testdir/test.toml' with OldTreeFilePath = "test.json" fwanalyzer will try to read 'testdir/test.json'. The -extra command line option can be used to overwrite the path: '-cfg testdir/test.toml -extra test1' will try to read 'test1/test.json'. Similar the newly generated filetree file will be stored in the same directory.

File modification check can be customized with:

Example:

[FileTreeCheck]
OldTreeFilePath       = "testtree.json"
CheckPath             = [ "/etc", "/bin" ]
CheckPermsOwnerChange = true
CheckFileSize         = true
CheckFileDigest       = false

Example Output:

"informational": {
    "/bin/bla": [ "CheckFileTree: new file: 40755 1001:1001 1024 0 SeLinux label: -" ]
}

Directory Content Check

The DirCheck (Directory content) check specifies a set of files that are allowed to be, or required to be, in a specified directory. Any other file or directory found in that directory will be reported as an offender. If an Allowed file isn't found, the check will pass. If a Required file is not found, it will be reported as an offender.

The file entries can contain wildcards like ?, *, and **. The allowed patterns are described in the golang documentation.

Only one DirCheck entry can exist per directory.

Example:

[DirContent."/home"]
Allowed = ["collin", "jon"]
Required = ["chris"]

Data Extract

The DataExtract option allows extracting data from a file and including it in the report. Data can be extracted via regular expression, by running an external script, or by reading a JSON object. The extracted data can later be used by the post processing script.

The Data Extract functionality adds the data to the report as a map of key:value pairs. The key is defined as the name of the statement or by the optional Name parameter. The value is the result of the regular expression or the output of the script.

Example: Regular expression based data extraction

The output generated by the regular expression will be stored as the value for the name of this statement, the example below is named "Version".

Example:

The key "Version" will contain the output of the regular expression.

[DataExtract."Version"]
File   = "/etv/versions"
RegEx  = ".*Ver=(.+)\n"
Desc   = "Ver 1337 test"

Example Output:

"data": {
  "Version": "1.2.3",
}

Example: Script-based data extraction

The output generated by the script will be stored as the value for the name of this statement, the example below is named LastLine.

The script is run with the following arguments:

<tmp filename> <original filename (fullpath)> <uid> <gid> <mode in octal> <selinux label or "-" for no label> [--] [script argument 1] ... [script argument N]

Example:

The key "script_test" will contain the output of the script. The name of this statement is "scripttest"

[DataExtract.scripttest]
File   = "/etc/somefile"
Script = "extractscripttest.sh"
Name   = "script_test"

Example Output:

"data": {
  "script_test": "some data",
}

Example: JSON data extraction

The output generated by the script will be stored as the value for the name of this statement, the example below is named LastLine.

Example:

The key "OS_Info" will containt the content of the Info field from the System object from /etc/os_version.json below.

{
  "System": {
    "Version": 7,
    "Arch": "arm32",
    "Info": "customized"
  }
}

[DataExtract.OS_Info]
File   = "/etc/os_version.json"
Json   = "System.Info"
Name   = "OSinfo"

Example Output:

"data": {
  "OSinfo": "customized",
}

Json arrays can be indexed by supplying the index instead of a field name.

Example: Advanced usage

The DataExtract statement allows multiple entries with the same Name (the same key). This can be useful for configuring multiple ways to extract the same information. The first data extract statement that produces valid output will set the value for the given key. This is supported for both regular expressions and scripts and a mixture of both.

The example below shows two statements that will both create the key value pair for the key "Version". If "1" does not produce valid output the next one is tried, in this case "2".

Example:

[DataExtract."1"]
File  = "/etc/versions"
RegEx = ".*Ver=(.+)\n"
Name  = "Version"

[DataExtract."2"]
File  = "/etc/OSVersion"
RegEx = ".*OS Version: (.+)\n"
Name  = "Version"

License

Copyright 2019-present, Cruise LLC

Licensed under the Apache License Version 2.0 (the "License"); you may not use this project except in compliance with the License.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Contributions

Contributions are welcome! Please see the agreement for contributions in CONTRIBUTING.md.

Commits must be made with a Sign-off (git commit -s) certifying that you agree to the provisions in CONTRIBUTING.md.