Awesome

IoC extractor

IoC extractor is an npm package for extracting common IoC (Indicator of Compromise) from a block of text.

Note: the package is highly influenced by cacador.

Installation

npm install -g ioc-extractor
# or if you want to use ioc-extractor as a library in your JS/TS project
npm install ioc-extractor

Usage

As a CLI

$ ioc-extractor --help
Usage: ioc-extractor [options]

Options:
  --no-strict            Disable strict option
  --no-refang            Disable refang option
  --no-sort              Disable sort option
  -p, --punycode         Enable punycode option
  -o, --only <types...>  Show only specific IoC types
  -h, --help             display help for command

$ echo "1.1.1.1 8.8.8.8 example.com" | ioc-extractor | jq
{
  "asns": [],
  "btcs": [],
  "cves": [],
  "domains": [
    "example.com"
  ],
  "emails": [],
  "eths": [],
  "gaPubIDs": [],
  "gaTrackIDs": [],
  "ipv4s": [
    "1.1.1.1",
    "8.8.8.8"
  ],
  "ipv6s": [],
  "macAddresses": [],
  "md5s": [],
  "sha1s": [],
  "sha256s": [],
  "sha512s": [],
  "ssdeeps": [],
  "urls": [],
  "xmrs": []
}

$ echo "1.1.1.1 8.8.8.8" | ioc-extractor --only ipv4s | jq
{
  "ipv4s": [
    "1.1.1.1",
    "8.8.8.8"
  ]
}

As a Library

import { extractIOC } from "ioc-extractor";

const input = "1.1.1[.]1 google(.)com f6f8179ac71eaabff12b8c024342109b";
const ioc = extractIOC(input);
console.log(ioc.md5s);
// => ['f6f8179ac71eaabff12b8c024342109b']
console.log(ioc.ipv4s);
// => ['1.1.1.1']
console.log(ioc.domains);
// => ['google.com']

extractIOC takes the following options:

punycode
refang
sort
strict

If you want to extract a specific type of IoC, you can use an extract function by IoC type.

import {
  refang,
  extractDomains,
  extractIPv4s,
  extractMD5s,
} from "ioc-extractor";

const input = "1.1.1[.]1 google(.)com f6f8179ac71eaabff12b8c024342109b";
const refanged = refang(input);
// => 1.1.1.1 google.com f6f8179ac71eaabff12b8c024342109b

const ipv4s = extractIPv4s(refanged);
// => ['1.1.1.1']

const domains = extractDomains(refanged);
// => ['google.com']

const md5s = extractMD5s(refanged);
// => ['f6f8179ac71eaabff12b8c024342109b']

Network related extract functions (e.g. extractDomains) can take the following options:

sort
strict

See docs for more details.

Alternatively, if you want to extract a list of specific IoC types at once, you can use partialExtractIOC.

import { partialExtractIOC } from "ioc-extractor";

const input = "1.1.1[.]1 google(.)com f6f8179ac71eaabff12b8c024342109b";
const ioc = partialExtractIOC(input, ["ipv4s", "domains"]);
console.log(ioc);
// => {"ipv4s":["1.1.1.1"],"domains":["google.com"]}

IoC Types

This package supports the following IoCs:

Hashes: MD5, SHA1, SHA256, SHA512, SSDEEP
Networks: domain, email, IPv4, IPv6, URL, ASN
Hardwares: MAC address
Utilities: CVE (CVE ID)
Cryptocurrencies: BTC (BTC address), ETH (ETH address), XMR (XMR address)
Trackers: GA track ID (Google Analytics tracking ID), GA pub ID (Google Adsense Publisher ID)

Refang Techniques

For Networks IoCs, the following refang techniques are supported:

Techniques	Defanged	Refanged
`.` in spaces	`1.1.1 . 1`	`1.1.1.1`
`.` in brackets, parentheses, etc.	`1.1.1[.]1`	`1.1.1.1`
`dot` in brackets, parentheses, etc.	`example[dot]com`	`example.com`
Back slash before `.`	`example\.com`	`example.com`
`/` in brackets, parentheses, etc.	`http://example.com[/]path`	`http://example.com/path`
`://` in brackets, parentheses, etc.	`http[://]example.com`	`http://example.com`
`:` in brackets, parentheses, etc.	`http[:]//example.com`	`http://example.com`
`@` in brackets, parentheses, etc.	`test[@]example.com`	`test@example.com`
`at` in brackets, parentheses, etc.	`test[at]example.com`	`test@example.com`
`hxxp`	`hxxps://example.com`	`https://example.com`
Partial	`1.1.1[.1`	`1.1.1.1`
Any combination	`hxxps[:]//test\.example[.)com[/]path`	`https://test.example.com/path`

Options

`strict`

Whether to do strict TLD matching or not. Defaults to true.

`refang`

Whether to do refang or not. Defaults to false.

`punycode`

Whether to do Punycode conversion or not. Defaults to false.

`sort`

Whether to sort values or not. Defaults to true.