Awesome

Cobalt Strike Beacon Dataset

https://research.nccgroup.com/2022/03/25/mining-data-from-cobalt-strike-beacons/

What is it?

Open Dataset containing 128340 rows of Cobalt Strike Beacon metadata as JSON-lines
Historical Beacon data spanning back to 2018
Team Server metadata (ip address, port, TLS certificate)
Enriched with GeoIP + ASN data
Original Beacon configuration block (deobfuscated) available in the config_block field
- Can be parsed using dissect.cobaltstrike or any other Beacon parser of choice.
All binary data fields are base64 encoded

How to load the data?

To overcome the filesize limit of 100mb on GitHub, the dataset is split by year as:

beacons-<year>.jsonl.gz

We recommend using Pandas to load the dataset. Example to load a single year:

>>> import pandas as pd
>>> df = pd.read_json("beacons-2022.jsonl.gz", lines=True)

Or to load everything as one big DataFrame:

>>> import pandas as pd
>>> from pathlib import Path
>>> gen_df = (pd.read_json(fname, lines=True) for fname in Path().glob("beacons-*.jsonl.gz"))
>>> df = pd.concat(gen_df, ignore_index=True)

See also our accompanying Jupyter notebook.

Exported Indicators of Compromise

We have also exported the IP and Host indicators from the Beacon dataset to a CSV file: iocs-export.csv.

You can find the CSV export code in the notebook. Note that actors can arbitrarily configure domain and host headers, and the IP address could be a CDN, such as Cloudflare. So, be careful when using this for blocking purposes.

Some handy tools for querying or exploring the CSV file:

good old grep (or git grep when you have this repo checked out)
VisiData (pip install visidata)
GitHub's Flat Viewer (flatgithub.com)

How does the data look like?

The dataset is a gzip compressed .jsonl file, i.e. each line is a JSON. Here is an example record:

$ zcat beacons-2018.jsonl.gz | head -n 1 | jq .
{
  "digest": {
    "md5": "d835cb78a451e0698e6b1a260bcc7d0f",
    "sha1": "c6bf7773eed78bc9c816856e168e8ba5275594a0",
    "sha256": "9be990a703227bee7be61c6713bda084a8f095eddf326c817ccdfa61454e76ab"
  },
  "filesize": 196608,
  "tls_subject": null,
  "tls_issuer": null,
  "tls_fingerprint": {
    "md5": null,
    "sha1": null,
    "sha256": null
  },
  "collected_from_ip": "217.182.71.175",
  "collected_from_port": 443,
  "collected_dt": "2018-07-04T00:00:00.000000",
  "domains": [
    "a0.awsstatic.com",
    "images.instagram.com",
    "media.tumblr.com",
    "cdn.zendesk.com"
  ],
  "max_setting_enum": 20,
  "beacon_version": "Cobalt Strike 3.5 (Sep 22, 2016)",
  "xorencoded": 0,
  "config_block": "AAEAAQACAAgAAgABAAIBuwADAAIABAAAAfQABAACAAQAEAAoAAUAAQACABQABgABAAIA/wAHAAMBADCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAg5WBcgWVRo+1nxVSVMT1ZJbBy57JeEIQjJIHZzw6n5BCPKGuwxjs17WPdLMIzY+osrrfFmB1N1pYZ29Lw+M9WyhyoNxPtD5MtFsP7z88oXzTZqOXgveD5dc4U9Jfbhv6tNGimS/JdmKBLF9OhoXkBOgryePBIbFz3IufvDTjX68CAwEAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAADAQBhMC5hd3NzdGF0aWMuY29tLC9fX3V0bS5naWYsaW1hZ2VzLmluc3RhZ3JhbS5jb20sL19fdXRtLmdpZixtZWRpYS50dW1ibHIuY29tLC9fX3V0bS5naWYsY2RuLnplbmRlc2suY29tLC9fX3V0bS5naWYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAkAAwCATW96aWxsYS81LjAgKGNvbXBhdGlibGU7IE1TSUUgOS4wOyBXaW5kb3dzIE5UIDYuMTsgV2luNjQ7IHg2NDsgVHJpZGVudC81LjApAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACgADAEAvX19fdXRtLmdpZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAwEAAAAABAAAAAIAAAAPAAAAAgAAAA8AAAACAAAACgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAMBAAAAAAkAAAASdXRtYWM9VUEtMjIwMjYwNC0yAAAACQAAAAd1dG1jbj0xAAAACQAAABB1dG1jcz1JU08tODg1OS0xAAAACQAAAA91dG1zcj0xMjgweDEwMjQAAAAJAAAADHV0bXNjPTMyLWJpdAAAAAkAAAALdXRtdWw9ZW4tVVMAAAAKAAAAI0hvc3Q6IGQydzZjNnhyd3NrMXpyLmNsb3VkZnJvbnQubmV0AAAABwAAAAAAAAAIAAAAAgAAAAZfX3V0bWEAAAAFAAAABXV0bWNjAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQADAQAAAAAKAAAAJkNvbnRlbnQtVHlwZTogYXBwbGljYXRpb24vb2N0ZXQtc3RyZWFtAAAABwAAAAAAAAACAAAABlVBLTIyMAAAAAEAAAACLTIAAAAFAAAABXV0bWFjAAAACQAAAAd1dG1jbj0xAAAACQAAABB1dG1jcz1JU08tODg1OS0xAAAACQAAAA91dG1zcj0xMjgweDEwMjQAAAAJAAAADHV0bXNjPTMyLWJpdAAAAAkAAAALdXRtdWw9ZW4tVVMAAAAKAAAAI0hvc3Q6IGQydzZjNnhyd3NrMXpyLmNsb3VkZnJvbnQubmV0AAAABwAAAAEAAAAEAAAAAAAAAAAAAA4AAwBAcnVuZGxsMzIuZXhlAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPAAMAgFxcJXNccGlwZVxtc2FnZW50XyV4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABMAAgAEAAAAAAAUAAIABAAAAAAAEAABAAIAAAARAAEAAgAAABIAAQACAAAAAA==",
  "cert_der": null,
  "xorkey": "0x69",
  "stage_prepend": null,
  "stage_append": null,
  "magic_pe": "UEU=",
  "magic_mz": "TVo=",
  "pe_compile_stamp": 1474078205,
  "pe_export_stamp": 1474078204,
  "SETTING_PROTOCOL": 8,
  "SETTING_PROTOCOL_TEXT": "BeaconProtocol.https",
  "SETTING_PORT": 443,
  "SETTING_SLEEPTIME": 500,
  "SETTING_MAXGET": 1048616,
  "SETTING_JITTER": 20,
  "SETTING_MAXDNS": 255,
  "SETTING_PUBKEY": "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCDlYFyBZVGj7WfFVJUxPVklsHLnsl4QhCMkgdnPDqfkEI8oa7DGOzXtY90swjNj6iyut8WYHU3Wlhnb0vD4z1bKHKg3E+0Pky0Ww/vPzyhfNNmo5eC94Pl1zhT0l9uG/q00aKZL8l2YoEsX06GheQE6CvJ48EhsXPci5+8NONfrwIDAQAB",
  "SETTING_PUBKEY_SHA256": "bc748561006feb7c46c34870a60ddf5c2b43ed56616c6a9f62a83114d7c76127",
  "SETTING_DOMAINS": "YTAuYXdzc3RhdGljLmNvbSwvX191dG0uZ2lmLGltYWdlcy5pbnN0YWdyYW0uY29tLC9fX3V0bS5naWYsbWVkaWEudHVtYmxyLmNvbSwvX191dG0uZ2lmLGNkbi56ZW5kZXNrLmNvbSwvX191dG0uZ2lm",
  "SETTING_USERAGENT": "TW96aWxsYS81LjAgKGNvbXBhdGlibGU7IE1TSUUgOS4wOyBXaW5kb3dzIE5UIDYuMTsgV2luNjQ7IHg2NDsgVHJpZGVudC81LjAp",
  "SETTING_SUBMITURI": "L19fX3V0bS5naWY=",
  "SETTING_C2_RECOVER": [
    "print=True",
    "prepend=15",
    "prepend=15",
    "prepend=10"
  ],
  "SETTING_C2_REQUEST": [
    "_PARAMETER=b'utmac=UA-2202604-2'",
    "_PARAMETER=b'utmcn=1'",
    "_PARAMETER=b'utmcs=ISO-8859-1'",
    "_PARAMETER=b'utmsr=1280x1024'",
    "_PARAMETER=b'utmsc=32-bit'",
    "_PARAMETER=b'utmul=en-US'",
    "_HEADER=b'Host: d2w6c6xrwsk1zr.cloudfront.net'",
    "BUILD=metadata",
    "NETBIOS=True",
    "PREPEND=b'__utma'",
    "PARAMETER=b'utmcc'"
  ],
  "SETTING_C2_POSTREQ": [
    "_HEADER=b'Content-Type: application/octet-stream'",
    "BUILD=metadata",
    "PREPEND=b'UA-220'",
    "APPEND=b'-2'",
    "PARAMETER=b'utmac'",
    "_PARAMETER=b'utmcn=1'",
    "_PARAMETER=b'utmcs=ISO-8859-1'",
    "_PARAMETER=b'utmsr=1280x1024'",
    "_PARAMETER=b'utmsc=32-bit'",
    "_PARAMETER=b'utmul=en-US'",
    "_HEADER=b'Host: d2w6c6xrwsk1zr.cloudfront.net'",
    "BUILD=output",
    "PRINT=True"
  ],
  "SETTING_SPAWNTO": "cnVuZGxsMzIuZXhl",
  "SETTING_PIPENAME": "XFwlc1xwaXBlXG1zYWdlbnRfJXg=",
  "SETTING_DNS_IDLE": 0,
  "SETTING_DNS_IDLE_IP": "0.0.0.0",
  "SETTING_DNS_SLEEP": 0,
  "SETTING_KILLDATE_YEAR": 0,
  "SETTING_KILLDATE_MONTH": 0,
  "SETTING_KILLDATE_DAY": 0,
  "country": "France",
  "country_code": "FR",
  "city": null,
  "longitude": 2.3387,
  "latitude": 48.8582,
  "asn": "16276",
  "org": "OVH SAS"
}