Awesome
<div align="center"> <p> <img alt="Hayabusa Logo" src="https://github.com/Yamato-Security/hayabusa/blob/main/logo.png" width="60%"> </p> [ <b>English</b> ] | [<a href="README-Japanese.md">日本語</a>] </div>About Hayabusa-Rules
This is a repository containing curated sigma rules that detect attacks in Windows event logs.
It is mainly used for Hayabusa detections rules and config files, as well as Velociraptor's built-in sigma detection.
The advantage of using this repository over the upstream sigma repository is that we include only rules that most sigma-native tools should be able to parse.
We also de-abstract the logsource
field by adding the necessary Channel
, EventID
, etc... fields to the rules to make it easier to understand what the rule is filtering on and more importantly to reduce false positives.
We also create new rules with converted field names and values for process_creation
rules and registry
based rules so that the sigma rules will not only detect on Sysmon logs, but will detect on built-in Windows logs as well.
Companion Projects
- EnableWindowsLogSettings - documentation and scripts to properly enable Windows event logs.
- Hayabusa - sigma-based threat hunting and fast forensics timeline generator for Windows event logs.
- Hayabusa Encoded Rules - The same as this repository but the rules and config files are stored in one file and XORed to prevent false positives from anti-virus.
- Hayabusa EVTX - A more maintained fork of the
evtx
crate. - Hayabusa Sample EVTXs - Sample evtx files to use for testing Hayabusa/Sigma detection rules.
- Presentations - Presentations from talks that we have given about our tools and resources.
- Sigma to Hayabusa Converter - Curates upstream Windows event log based Sigma rules into an easier to use form.
- Takajo - Analyzer for hayabusa results.
- WELA (Windows Event Log Analyzer) - An analyzer for Windows event logs written in PowerShell. (Deprecated and replaced by Takajo.)
Table of Contents
- About Hayabusa-Rules
- Companion Projects
- Table of Contents
- About creating rule files
- Abbreviations
- Detection field
- Sigma correlations
- Deprecated features
- Rule creation advice
- Converting Sigma rules to Hayabusa format
About creating rule files
Hayabusa detection rules are written in YAML format with a file extension of .yml
. (.yaml
files will be ignored.)
They are a subset of sigma rules but also contain some added features.
We are trying to make them as close to sigma rules as possible so that it is easy to convert Hayabusa rules back to sigma to give back to the community.
Hayabusa rules can express complex detection rules by combining not only simple string matching but also regular expressions, AND
, OR
, and other conditions.
In this section, we will explain how to write Hayabusa detection rules.
Rule file format
Example:
#Author section
author: Zach Mathis
date: 2022/03/22
modified: 2022/04/17
#Alert section
title: Possible Timestomping
details: 'Path: %TargetFilename% ¦ Process: %Image% ¦ User: %User% ¦ CreationTime: %CreationUtcTime% ¦ PreviousTime: %PreviousCreationUtcTime% ¦ PID: %PID% ¦ PGUID: %ProcessGuid%'
description: |
The Change File Creation Time Event is registered when a file creation time is explicitly modified by a process.
This event helps tracking the real creation time of a file.
Attackers may change the file creation time of a backdoor to make it look like it was installed with the operating system.
Note that many processes legitimately change the creation time of a file; it does not necessarily indicate malicious activity.
#Rule section
id: f03e34c4-6432-4a30-9ae2-76ae6329399a
level: low
status: stable
logsource:
product: windows
service: sysmon
definition: Sysmon needs to be installed and configured.
detection:
selection_basic:
Channel: Microsoft-Windows-Sysmon/Operational
EventID: 2
condition: selection_basic
falsepositives:
- unknown
tags:
- t1070.006
- attack.defense_evasion
references:
- https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon
- https://attack.mitre.org/techniques/T1070/006/
ruletype: Hayabusa
#Sample XML Event
sample-message: |
File creation time changed:
RuleName: technique_id=T1099,technique_name=Timestomp
UtcTime: 2022-04-12 22:52:00.688
ProcessGuid: {43199d79-0290-6256-3704-000000001400}
ProcessId: 9752
Image: C:\TMP\mim.exe
TargetFilename: C:\Users\IEUser\AppData\Local\Temp\Quest Software\PowerGUI\51f5c69c-5d16-47e1-9864-038c8510d919\mk.ps1
CreationUtcTime: 2016-05-16 09:13:50.950
PreviousCreationUtcTime: 2022-04-12 22:52:00.563
User: ZACH-LOG-TEST\IEUser
sample-evtx: |
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Sysmon" Guid="{5770385f-c22a-43e0-bf4c-06f5698ffbd9}" />
<EventID>2</EventID>
<Version>5</Version>
<Level>4</Level>
<Task>2</Task>
<Opcode>0</Opcode>
<Keywords>0x8000000000000000</Keywords>
<TimeCreated SystemTime="2022-04-12T22:52:00.689654600Z" />
<EventRecordID>8946</EventRecordID>
<Correlation />
<Execution ProcessID="3408" ThreadID="4276" />
<Channel>Microsoft-Windows-Sysmon/Operational</Channel>
<Computer>Zach-log-test</Computer>
<Security UserID="S-1-5-18" />
</System>
<EventData>
<Data Name="RuleName">technique_id=T1099,technique_name=Timestomp</Data>
<Data Name="UtcTime">2022-04-12 22:52:00.688</Data>
<Data Name="ProcessGuid">{43199d79-0290-6256-3704-000000001400}</Data>
<Data Name="ProcessId">9752</Data>
<Data Name="Image">C:\TMP\mim.exe</Data>
<Data Name="TargetFilename">C:\Users\IEUser\AppData\Local\Temp\Quest Software\PowerGUI\51f5c69c-5d16-47e1-9864-038c8510d919\mk.ps1</Data>
<Data Name="CreationUtcTime">2016-05-16 09:13:50.950</Data>
<Data Name="PreviousCreationUtcTime">2022-04-12 22:52:00.563</Data>
<Data Name="User">ZACH-LOG-TEST\IEUser</Data>
</EventData>
</Event>
Author section
- author [required]: Name of the author(s).
- date [required]: Date the rule was made.
- modified [optional]: Date the rule was updated.
Alert section
- title [required]: Rule file title. This will also be the name of the alert that gets displayed so the briefer the better. (Should not be longer than 85 characters.)
- details [optional]: The details of the alert that gets displayed. Please output any fields in the Windows event log that are useful for analysis. Fields are seperated by
" ¦ "
. Field placeholders are enclosed with a%
(Example:%MemberName%
) and need to be defined inrules/config/eventkey_alias.txt
. (Explained below.) - description [optional]: A description of the rule. This does not get displayed so you can make this long and detailed.
Rule section
- id [required]: A randomly generated version 4 UUID used to uniquely identify the rule. You can generate one here.
- level [required]: Severity level based on sigma's definition. Please write one of the following:
informational
,low
,medium
,high
,critical
- status[required]: Status based on sigma's definition. Please write one of the following:
deprecated
,experimental
,test
,stable
. - logsource [required]: While this is not actually used by Hayabusa at the moment, we define logsource in the same way as sigma in order to be compatible with sigma rules.
- detection [required]: The detection logic goes here. (Explained below.)
- falsepositives [required]: The possibilities for false positives. For example:
system administrator
,normal user usage
,normal system usage
,legacy application
,security team
,none
. If it is unknown, please writeunknown
. - tags [optional]: If the technique is a LOLBINS/LOLBAS technique, please add the
lolbas
tag. If the alert can be mapped to a technique in the MITRE ATT&CK framework, please add the tactic ID (Example:attack.t1098
) and any applicable tactics below:attack.reconnaissance
-> Reconnaissance (Recon)attack.resource_development
-> Resource Development (ResDev)attack.initial_access
-> Initial Access (InitAccess)attack.execution
-> Execution (Exec)attack.persistence
-> Persistence (Persis)attack.privilege_escalation
-> Privilege Escalation (PrivEsc)attack.defense_evasion
-> Defense Evasion (Evas)attack.credential_access
-> Credential Access (CredAccess)attack.discovery
-> Discovery (Disc)attack.lateral_movement
-> Lateral Movement (LatMov)attack.collection
-> Collection (Collect)attack.command_and_control
-> Command and Control (C2)attack.exfiltration
-> Exfiltration (Exfil)attack.impact
-> Impact (Impact)
- references [optional]: Any links to references.
- ruletype [required]:
Hayabusa
for hayabusa rules. Rules automatically converted from sigma Windows rules will beSigma
.
Sample XML Event
- sample-message [required]: Starting forward, we ask rule authors to include sample messages for their rules. This is the rendered message that Windows' Event Viewer displays.
- sample-evtx [required]: Starting forward, we ask rule authors to include sample XML events for their rules.
Abbreviations
The following abbreviations are used in rules in order to make the output as concise as possible:
Acct
-> AccountAddr
-> AddressAuth
-> AuthenticationCli
-> ClientChan
-> ChannelCmd
-> CommandCnt
-> CountComp
-> ComputerConn
-> Connection/ConnectedCreds
-> CredentialsCrit
-> CriticalDisconn
-> Disconnection/DisconnectedDir
-> DirectoryDrv
-> DriverDst
-> DestinationEID
-> Event IDErr
-> ErrorExec
-> ExecutionFP
-> False PositiveFW
-> FirewallGTW
-> GatewayGrp
-> GroupImg
-> ImageInj
-> InjectionKrb
-> KerberosLID
-> Logon IDMed
-> MediumNet
-> NetworkObj
-> ObjectOp
-> Operational/OperationProto
-> ProtocolPW
-> PasswordReconn
-> ReconnectionReq
-> RequestRsp
-> ResponseSess
-> SessionSig
-> SignatureSusp
-> SuspiciousSrc
-> SourceSvc
-> ServiceSvr
-> ServerTemp
-> TemporaryTerm
-> Termination/TerminatedTkt
-> TicketTgt
-> TargetUnkwn
-> UnknownUsr
-> UserPerm
-> PermamentPkg
-> PackagePriv
-> PrivilegeProc
-> ProcessPID
-> Process IDPGUID
-> Process GUID (Global Unique ID)Ver
-> Version
Detection field
Selection fundamentals
First, the fundamentals of how to create a selection rule will be explained.
How to write AND and OR logic
To write AND logic, we use nested dictionaries. The detection rule below defines that both conditions have to be true in order for the rule to match.
- EventID has to exactly be
7040
. - AND
- Channel has to exactly be
System
.
detection:
selection:
Event.System.EventID: 7040
Event.System.Channel: System
condition: selection
To write OR logic, we use lists (Dictionaries that start with -
).
In the detection rule below, either one of the conditions will result in the rule being triggered.
- EventID has to exactly be
7040
. - OR
- Channel has to exactly be
System
.
detection:
selection:
- Event.System.EventID: 7040
- Event.System.Channel: System
condition: selection
We can also combine AND
and OR
logic as shown below.
In this case, the rule matches when the following two conditions are both true.
- EventID is either exactly
7040
OR7041
. - AND
- Channel is exactly
System
.
detection:
selection:
Event.System.EventID:
- 7040
- 7041
Event.System.Channel: System
condition: selection
Eventkeys
The following is an excerpt of a Windows event log, formatted in the original XML.
The Event.System.Channel
field in the rule file example above refers to the original XML tag: <Event><System><Channel>System<Channel><System></Event>
Nested XML tags are replaced by tag names seperated by dots (.
).
In hayabusa rules, these field strings connected together with dots are refered to as eventkeys
.
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>
<System>
<EventID>7040</EventID>
<Channel>System</Channel>
</System>
<EventData>
<Data Name='param1'>Background Intelligent Transfer Service</Data>
<Data Name='param2'>auto start</Data>
</EventData>
</Event>
Eventkey Aliases
Long eventkeys with many .
seperations are common, so hayabusa will use aliases to make them easier to work with. Aliases are defined in the rules/config/eventkey_alias.txt
file. This file is a CSV file made up of alias
and event_key
mappings. You can rewrite the rule above as shown below with aliases making the rule easier to read.
detection:
selection:
Channel: System
EventID: 7040
condition: selection
Caution: Undefined Eventkey Aliases
Not all eventkey aliases are defined in rules/config/eventkey_alias.txt
. If you are not getting the correct data in the details
(Alert details
) message, and instead are getting n/a
(not available) or if the selection in your detection logic is not working properly, then you may need to update rules/config/eventkey_alias.txt
with a new alias.
How to use XML attributes in conditions
XML elements may have attributes set by adding a space to the element. For example, Name
in Provider Name
below is an XML attribute of the Provider
element.
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>
<System>
<Provider Name='Microsoft-Windows-Security-Auditing' Guid='{54849625-5478-4994-a5ba-3e3b0328c30d}'/>
<EventID>4672</EventID>
<EventRecordID>607469</EventRecordID>
<Channel>Security</Channel>
<Security />
</System>
</Event>
To specify XML attributes in an eventkey, use the format {eventkey}_attributes.{attribute_name}
. For example, to specify the Name
attribute of the Provider
element in a rule file, it would look like this:
detection:
selection:
Channel: Security
EventID: 4672
Event.System.Provider_attributes.Name: 'Microsoft-Windows-Security-Auditing'
condition: selection
grep search
Hayabusa can perform grep searches in Windows event log files by not specifying any eventkeys.
To do a grep search, specify the detection as shown below. In this case, if the strings mimikatz
or metasploit
are included in the Windows Event log, it will match. It is also possible to specify wildcards.
detection:
selection:
- mimikatz
- metasploit
Note: Hayabusa internally converts Windows event log data to JSON format before processing the data so it is not possible to match on XML tags.
EventData
Windows event logs are divided into two parts: the System
part where the fundamental data (Event ID, Timestamp, Record ID, Log name (Channel)) is written, and the EventData
or UserData
part where arbitrary data is written depending on the Event ID.
One problem that arises often is that the names of the fields nested in EventData
are all called Data
so the eventkeys described so far cannot distinguish between SubjectUserSid
and SubjectUserName
.
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>
<System>
<EventID>5379</EventID>
<TimeCreated SystemTime='2021-10-20T10:16:18.7782563Z' />
<EventRecordID>607469</EventRecordID>
<Channel>Security</Channel>
<Security />
</System>
<EventData>
<Data Name='SubjectUserSid'>S-1-1-11-1111111111-111111111-1111111111-1111</Data>
<Data Name='SubjectUserName'>hayabusa</Data>
<Data Name='SubjectDomainName'>DESKTOP-HAYABUSA</Data>
<Data Name='SubjectLogonId'>0x11111111</Data>
</EventData>
</Event>
To deal with this problem, you can specify the value assigned in Data Name
. For example, if you want to use SubjectUserName
and SubjectDomainName
in the EventData as a condition of a rule, you can describe it as follows:
detection:
selection:
Channel: System
EventID: 7040
Event.EventData.SubjectUserName: hayabusa
Event.EventData.SubjectDomainName: DESKTOP-HAYBUSA
condition: selection
Abnormal patterns in EventData
Some of the tags nested in EventData
do not have a Name
attribute.
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>
<System>
<EventID>5379</EventID>
<Channel>Security</Channel>
<Security />
</System>
<EventData>
<Data>Available</Data>
<Data>None</Data>
<Data>NewEngineState=Available PreviousEngineState=None (...)</Data>
</EventData>
</Event>
To detect an event log like the one above, you can specify an eventkey named Data
.
In this case, the condition will match as long as any one of the nested Data
tags equals None
.
detection:
selection:
Channel: Security
EventID: 5379
Data: None
condition: selection
Outputting field data from multiple field names with the same name
Some events will save their data to field names all called Data
like in the previous example.
If you specify %Data%
in details:
, all of the data will be outputted in an array.
For example:
["rundll32.exe","6.1.7600.16385","4a5bc637","KERNELBASE.dll","6.1.7601.23392","56eb2fb9","c0000005"]
If you want to print out just the first Data
field data, you can specify %Data[1]%
in your details:
alert string and only rundll32.exe
will be outputted.
Field Modifiers
A pipe character can be used with eventkeys as shown below for matching strings.
All of the conditions we have described so far use exact matches, but by using field modifiers, you can describe more flexible detection rules.
In the following example, if a value of Data
contains the string EngineVersion=2
, it will match the condition.
detection:
selection:
Channel: 'Windows PowerShell'
EventID: 400
Data|contains: 'EngineVersion=2'
condition: selection
String matches are case insensitive. However, they become case sensitive whenever |re
or |equalsfield
are used.
Supported Sigma Field Modifiers
You can check the current status of all of the supported and unsupported field modifiers as well as how many times these modifiers are used in Sigma and Hayabusa rules at https://github.com/Yamato-Security/hayabusa-rules/blob/main/doc/SupportedSigmaFieldModifiers.md . This document is updated every time there is an update to Sigma or Hayabusa rules.
-
'|all':
: This field modifier is different from those above because it does not get applied to a certain field but to all fields.In this example, both strings
Keyword-1
andKeyword-2
need to exist but can exist anywhere in any field:detection: keywords: '|all': - 'Keyword-1' - 'Keyword-2' condition: keywords
-
|base64offset|contains
: Data will be encoded to base64 in three different ways depending on its position in the encoded string. This modifier will encoded a string to all three variations and check if the string is encoded somewhere in the base64 string. -
|cased
: Makes the search case-sensitive. -
|cidr
: Checks if a field value matches on a IPv4 or IPv6 CIDR notation. (Ex:192.0.2.0/24
) -
|contains
: Checks if a field value contains a certain string. -
|contains|all
: Checks if multiple words are contained in the data. -
|contains|all|windash
: Same as|contains|windash
but all of the keywords need to be present. -
|contains|cased
: Checks if a field value contains a certain case-sensitive string. -
|contains|windash
: Will check the string as-is, as well as convert the first-
character to/
,–
(en dash),—
(em dash), and―
(horizontal bar) character permutations. -
|endswith
: Checks if a field value ends with a certain string. -
|endswith|cased
: Checks if a field value ends with a certain case-sensitive string. -
|endswith|windash
: Checks the end of the string and performs variations for dashes. -
|exists
: Checks if a field exists. -
|fieldref
: Checks to see if the values in two fields are the same. You can usenot
in thecondition
if you want to check if two fields are different. -
|fieldref|contains
: Checks to see if the value of one field is contained in another field. -
|fieldref|endswith
: Check if the field on the left ends with the string of the field on the right. You can usenot
in thecondition
to check if they are different. -
|fieldref|startswith
: Check if the field on the left starts with the string of the field on the right. You can usenot
in thecondition
to check if they are different. -
|gt
: Checks if a field value is greater than a certain number. -
|gte
: Checks if a field value is greater than or equal to a certain number. -
|lt
: Checks if a field value is less than a certain number. -
|lte
: Checks if a field value is less than or equal to a certain number. -
|re
: Use case-sensitive regular expressions. (We are using the regex crate so please out the documentation at https://docs.rs/regex/latest/regex/#syntax to learn how to write supported regular expressions.)Caution: Regular expression syntax in Sigma rules uses PCRE with certain metacharacters for character classes, lookbehind, atomic grouping, etc... being unsupported. The Rust regex crate should be able to use all regular expressions in Sigma rules but there is a possibility of incompatibility.
-
|re|i
: (Insensitive) Use case-insensitive regular expressions. -
|re|m
: (Multi-line) Match across multiple lines.^
/$
match the start/end of line. -
|re|s
: (Single-line) dot (.
) matches all characters, including the newline character. -
|startswith
: Checks if a field value starts with a certain string. -
|startswith|cased
: Checks if a field value starts with a certain case-sensitive string. -
|utf16|base64offset|contains
: Checks to see if a certain UTF-16 string is encoded inside a base64 string. -
|utf16be|base64offset|contains
: Checks to see if a certain UTF-16 big-endian string is encoded inside a base64 string. -
|utf16le|base64offset|contains
: Checks to see if a certain UTF-16 little-endian string is encoded inside a base64 string. -
|wide|base64offset|contains
: Alias forutf16le|base64offset|contains
, checking for UTF-16 little-endian strings.
Deprecated Field Modifiers
The following modifiers are now deprecated and replaced by modifiers that adhere more to the sigma specifications.
|equalsfield
: Now is replaced by|fieldref
.|endswithfield
: Now is replaced by|fieldref|endswith
.
Unsupported Field Modifiers
The following modifiers are currently not supported:
contains|expand
expand
Wildcards
Wildcards can be used in eventkeys. In the example below, if ProcessCommandLine
starts with the string "malware", the rule will match.
The specification is fundamentally the same as sigma rule wildcards so will be case insensitive.
detection:
selection:
Channel: Security
EventID: 4688
ProcessCommandLine: malware*
condition: selection
The following two wildcards can be used.
*
: Matches any string of zero or more characters. (Internally it is converted to the regular expression.*
)?
: Matches any single character. (Internally converted to the regular expression.
)
About escaping wildcards:
- Wildcards (
*
and?
) can be escaped by using a backslash:\*
,\?
. - If you want to use a backslash right before a wildcard then write
\\*
or\\?
. - Escaping is not required if you are using backslashes by themselves.
null keyword
The null
keyword can be used to check if field does not exist.
detection:
selection:
EventID: 4688
ProcessCommandLine: null
condition: selection
Note: This is different from ProcessCommandLine: ''
which checks if the value of a field is empty.
condition
With the notation we explained above, you can express AND
and OR
logic but it will be confusing if you are trying to define complex logic.
When you want to make more complex rules, you should use the condition
keyword as shown below.
detection:
SELECTION_1:
EventID: 3
SELECTION_2:
Initiated: 'true'
SELECTION_3:
DestinationPort:
- '4444'
- '666'
SELECTION_4:
Image: '*\Program Files*'
SELECTION_5:
DestinationIp:
- 10.*
- 192.168.*
- 172.16.*
- 127.*
SELECTION_6:
DestinationIsIpv6: 'false'
condition: (SELECTION_1 and (SELECTION_2 and SELECTION_3) and not ((SELECTION_4 or (SELECTION_5 and SELECTION_6))))
The following expressions can be used for condition
.
{expression1} and {expression2}
: Require both {expression1} AND {expression2}{expression1} or {expression2}
: Require either {expression1} OR {expression2}not {expression}
: Reverse the logic of {expression}( {expression} )
: Set precedance of {expression}. It follows the same precedance logic as in mathematics.
In the above example, selection names such as SELECTION_1
, SELECTION_2
, etc... are used but they can be named anything as long as they only contain the following characters: a-z A-Z 0-9 _
However, please use the standard convention of
selection_1
,selection_2
,filter_1
,filter_2
, etc... to make things easy to read whenever possible.
not logic
Many rules will result in false positives so it is very common to have a selection for signatures to search for but also a filter selection to not alert on false positives. For example:
detection:
selection:
Channel: Security
EventID: 4673
filter:
- ProcessName: C:\Windows\System32\net.exe
- ProcessName: C:\Windows\System32\lsass.exe
- ProcessName: C:\Windows\System32\audiodg.exe
- ProcessName: C:\Windows\System32\svchost.exe
- ProcessName: C:\Windows\System32\mmc.exe
- ProcessName: C:\Windows\System32\net.exe
- ProcessName: C:\Windows\explorer.exe
- ProcessName: C:\Windows\System32\SettingSyncHost.exe
- ProcessName: C:\Windows\System32\sdiagnhost.exe
- ProcessName|startswith: C:\Program Files
- SubjectUserName: LOCAL SERVICE
condition: selection and not filter
Sigma correlations
We have implemented half of the Sigma version 2 correlations as defined here.
Supported correlations:
- Event Count (
event_count
) - Value Count (
value_count
)
Unsupported correlations:
- Temporal Proximity (
temporal
) - Ordered Temporal Proximity (
temporal_ordered
)
Event Count rules
These are rules that count certain events and alert if too many or not enough number of these events occur within a timeframe. Common examples of detecting many events within a certain time period are for detecting password guessing attacks, password spray attacks and denial of service attacks. You could also use these rules to detect log source reliability issues, such as when certain events fall below a certain threshold.
Event Count rule example:
The following example uses two rules to detect password guessing attacks.
There will be an alert when the referenced rule matches 5 or more times within 5 minutes and the IpAddress
field is the same for those events.
Note that we have only included the necessary fields in order to understand the concept. The full rule that this example is based on is located here for your reference.
Event Count correlation rule:
title: PW Guessing
id: 23179f25-6fce-4827-bae1-b219deaf563e
correlation:
type: event_count
rules:
- 5b0b75dc-9190-4047-b9a8-14164cee8a31
group-by:
- IpAddress
timespan: 5m
condition:
gte: 5
Failed Logon - Incorrect Password rule:
title: Failed Logon - Incorrect Password
id: 5b0b75dc-9190-4047-b9a8-14164cee8a31
logsource:
product: windows
service: security
detection:
selection:
Channel: Security
EventID: 4625
SubStatus: "0xc000006a" #Wrong password
filter:
IpAddress: "-"
condition: selection and not filter
Deprecated count
rule example:
The above correlation and referenced rules provide the same results as the following rule which uses the older count
modifier:
title: PW Guessing
logsource:
product: windows
service: security
detection:
selection:
Channel: Security
EventID: 4625
SubStatus: "0xc000006a" #Wrong password
filter:
IpAddress: "-"
condition: selection and not filter | count() by IpAddress >= 5
timeframe: 5m
Event Count rule output:
The rules above will create the following output:
% ./hayabusa csv-timeline -d ../hayabusa-sample-evtx -r password-guessing-sample.yml -w
%
Timestamp · RuleTitle · Level · Computer · Channel · EventID · RecordID · Details · ExtraFieldInfo
2016-09-20 01:50:06.513 +09:00 · PW Guessing · med · DESKTOP-M5SN04R · Sec · 4625 · - · Count: 3558 ¦ IpAddress: 192.168.198.149 · -
Value Count rules
These rules counts the same events within a time frame with different values of a given field.
Examples:
- Network scans where a single source IP address tries to connect to many different destination IP addresses and/or ports.
- Password spraying attacks where a single source fails to authenticate with many different users.
- Detect tools like BloodHound that enumerate many high-privilege AD groups within a short time frame.
Value Count rule example:
The following rule detects when an attacker is trying to guess usernames.
That is, when the same source IP address (IpAddress
) fails to logon with more than 3 different usernames (TargetUserName
) within 5 minutes.
Note that we have only included the necessary fields in order to understand the concept. The full rule that this example is based on is located here for your reference.
Value Count correlation rule:
title: User Guessing
id: 0ae09af3-f30f-47c2-a31c-83e0b918eeee
correlation:
type: value_count
rules:
- b2c74582-0d44-49fe-8faa-014dcdafee62
group-by:
- IpAddress
timespan: 5m
condition:
gt: 3
field: TargetUserName
Value Count Logon Failure (Non-existant User) rule:
title: Failed Logon - Non-Existant User
id: b2c74582-0d44-49fe-8faa-014dcdafee62
logsource:
product: windows
service: security
detection:
selection:
Channel: Security
EventID: 4625
SubStatus: "0xc0000064" #Username does not exist
condition: selection
Deprecated count
modifier rule:
The above correlation and referenced rules provide the same results as the following rule which uses the older count
modifier:
title: User Guessing
logsource:
product: windows
service: security
detection:
selection:
Channel: Security
EventID: 4625
SubStatus: "0xc0000064" #Username does not exist
condition: selection | count(TargetUserName) by IpAddress > 3
timeframe: 5m
Value Count rule output:
The rules above will create the following output:
2018-08-23 23:24:22.523 +09:00 · User Guessing · med · dmz-ftp · Sec · 4625 · - · Count: 4 ¦ TargetUserName: ninja-labs/root/test@ninja-labs.com/sarutobi ¦ IpAddress: - ¦ LogonType: 8 ¦ TargetDomainName: ¦ ProcessName: C:\\Windows\\System32\\svchost.exe ¦ LogonProcessName: Advapi ¦ WorkstationName: DMZ-FTP · -
2018-08-28 08:03:13.770 +09:00 · User Guessing · med · dmz-ftp · Sec · 4625 · - · Count: 4 ¦ TargetUserName: root/sarutobi@ninja-labs.com/sarutobi/administrator@ninja-labs.com ¦ IpAddress: - ¦ LogonType: 8 ¦ TargetDomainName: ¦ ProcessName: C:\\Windows\\System32\\svchost.exe ¦ LogonProcessName: Advapi ¦ WorkstationName: DMZ-FTP · -
2018-09-01 12:51:58.346 +09:00 · User Guessing · med · dmz-ftp · Sec · 4625 · - · Count: 4 ¦ TargetUserName: root/admin@ninja-labs.com/admin/administrator@ninja-labs.com ¦ IpAddress: - ¦ LogonType: 8 ¦ TargetDomainName: ¦ ProcessName: C:\\Windows\\System32\\svchost.exe ¦ LogonProcessName: Advapi ¦ WorkstationName: DMZ-FTP · -
2018-09-02 03:55:13.007 +09:00 · User Guessing · med · dmz-ftp · Sec · 4625 · - · Count: 4 ¦ TargetUserName: root/admin@ninja-labs.com/administrator@ninja-labs.com/admin ¦ IpAddress: - ¦ LogonType: 8 ¦ TargetDomainName: ¦ ProcessName: C:\\Windows\\System32\\svchost.exe ¦ LogonProcessName: Advapi ¦ WorkstationName: DMZ-FTP · -
Notes on correlation rules
-
You should include all of your correlation and referenced rules in a single file and separate them with a YAML separator of
---
. -
By default, referenced correlation rules will not be outputted. If you want to see the output of the referenced rules, then you need to add
generate: true
undercorrelation
. This is very useful to turn on and check when creating correlation rules.Example:
correlation: generate: true
-
You can use alias names instead of rule IDs when referencing rules in order to make things easier to understand.
-
You can reference multiple rules.
-
You can use multiple fields in
group-by
. If you do, then all of the values in those fields need to be the same or else you will not get an alert. Most of the time, you will write rules that filter on certain fields withgroup-by
in order to reduce false positives, however, it is possible to omitgroup-by
to create a more generic rule. -
The timestamp of the correlation rule will be the very beginning of the attack so you should check events after that to confirm if it is a false positive or not.
Deprecated features
These features are still supported in Hayabusa but will not be used inside rules in the future.
Nesting keywords inside eventkeys
Eventkeys can be nested with specific keywords. In the example below, the rule will match if the following are true:
ServiceName
is calledmalicious-service
or contains a regular expression in./rules/config/regex/detectlist_suspicous_services.txt
.ImagePath
has a minimum of 1000 characters.ImagePath
does not have any matches in theallowlist
.
detection:
selection:
Channel: System
EventID: 7045
ServiceName:
- value: malicious-service
- regexes: ./rules/config/regex/detectlist_suspicous_services.txt
ImagePath:
min_length: 1000
allowlist: ./rules/config/regex/allowlist_legitimate_services.txt
condition: selection
Currently, the following keywords can be specified:
value
: matches by string (wildcards and pipes can also be specified).min_length
: matches when the number of characters is greater than or equal to the specified number.regexes
: matches if one of the regular expressions in the file that you specify in this field matches.allowlist
: rule will be skipped if there is any match found in the list of regular expressions in the file that you specify in this field.
regexes and allowlist keywords
Hayabusa had two built-in regular expression files used for the ./rules/hayabusa/default/alerts/System/7045_CreateOrModiftySystemProcess-WindowsService_MaliciousServiceInstalled.yml
file:
./rules/config/regex/detectlist_suspicous_services.txt
: to detect suspicious service names./rules/config/regex/allowlist_legitimate_services.txt
: to allow legitimate services
Files defined in regexes
and allowlist
can be edited to change the behavior of all rules that reference them without having to change any rule file itself.
You can also use different detectlist and allowlist textfiles that you create.
Aggregation conditions (Count rules)
This is still supported in Hayabusa but will be replaced by Sigma correlation rules in the future.
Basics
The condition
keyword described above implements not only AND
and OR
logic, but is also able to count or "aggregate" events.
This function is called the "aggregation condition" and is specified by connecting a condition with a pipe.
In this password spray detection example below, a conditional expression is used to determine if there are 5 or more TargetUserName
values from one source IpAddress
within a time frame of 5 minutes.
detection:
selection:
Channel: Security
EventID: 4648
condition: selection | count(TargetUserName) by IpAddress > 5
timeframe: 5m
Aggregation conditions can be defined in the following format:
count() {operator} {number}
: For log events that match the first condition before the pipe, the condition will match if the number of matched logs satisfies the condition expression specified by{operator}
and{number}
.
{operator}
can be one of the following:
==
: If the value is equal to the specified value, it is treated as matching the condition.>=
: If the value is greater than or equal to the specified value, the condition is considered to have been met.>
: If the value is greater than the specified value, the condition is considered to have been met.<=
: If the value is less than or equal to the specified value, the condition is considered to have been met.<
: If the value is less than the specified value, it will be treated as if the condition is met.
{number}
must be a number.
timeframe
can be defined in the following:
15s
: 15 seconds30m
: 30 minutes12h
: 12 hours7d
: 7 days3M
: 3 months
Four patterns for aggregation conditions
- No count argument or
by
keyword. Example:selection | count() > 10
If
selection
matches more than 10 times within the time frame, the condition will match. These are replaced by Event Count correlation rules that do not use thegroup-by
field. - No count argument but there is a
by
keyword. Example:selection | count() by IpAddress > 10
selection
will have to be true more than 10 times for the sameIpAddress
. These #2 rules are more common than the #1 rules. You can also specify multiple fields to group by. For example:by IpAddress, Computer
These are replaced by Event Count correlation rules that do use thegroup-by
field. - There is a count argument but no
by
keyword. Example:selection | count(TargetUserName) > 10
If
selection
matches andTargetUserName
is different more than 10 times within the time frame, the condition will match. These are replaced by Value Count correlation rules that do not use thegroup-by
field. - There is both a count argument and
by
keyword. Example:selection | count(Users) by IpAddress > 10
For the same
IpAddress
, there will need to be more than 10 differentTargetUserName
in order for the condition to match. These #4 rules are more common than the #3 rules. These are replaced by Value Count correlation rules that use thegroup-by
field.
Pattern 1 example
This is the most basic pattern: count() {operator} {number}
. The rule below will match if selection
happens 3 or more times.
Pattern 2 example
count() by {eventkey} {operator} {number}
: Log events that match the condition
before the pipe are grouped by the same {eventkey}
. If the number of matched events for each grouping satisfies the condition specified by {operator}
and {number}
, then the condition will match.
Pattern 3 example
count({eventkey}) {operator} {number}
: Counts how many different values of {eventkey}
exist in the log event that match the condition before the condition pipe. If the number satisfies the conditional expression specified in {operator}
and {number}
, the condition is considered to have been met.
Pattern 4 example
count({eventkey_1}) by {eventkey_2} {operator} {number}
: The logs that match the condition before the condition pipe are grouped by the same {eventkey_2}
, and the number of different values of {eventkey_1}
in each group is counted. If the values counted for each grouping satisfy the conditional expression specified by {operator}
and {number}
, the condition will match.
Count rule output
The details output for count rules is fixed and will print the original count condition in [condition]
followed by the recorded eventkeys in [result]
.
In the example below, a list of TargetUserName
usernames that were being bruteforced followed by the source IpAddress
:
[condition] count(TargetUserName) by IpAddress >= 5 in timeframe [result] count:41 TargetUserName:jorchilles/jlake/cspizor/lpesce/bgalbraith/jkulikowski/baker/eskoudis/dpendolino/sarmstrong/lschifano/drook/rbowes/ebooth/melliott/econrad/sanson/dmashburn/bking/mdouglas/cragoso/psmith/bhostetler/zmathis/thessman/kperryman/cmoody/cdavis/cfleener/gsalinas/wstrzelec/jwright/edygert/ssims/jleytevidal/celgee/Administrator/mtoussain/smisenar/tbennett/bgreenwood IpAddress:10.10.2.22 timeframe:5m
The timestamp of the alert will be the time from the first event detected.
Rule creation advice
-
When possible, always specify the
Channel
orProviderName
name and theEventID
number. By default, only the event IDs listed in./rules/config/target_event_IDs.txt
will be scanned so you may need to add a newEventID
number to this file if the EID is not already in there. -
Please do not use multiple
selection
orfilter
fields and excessive grouping when it is not needed. For example:
Instead of
detection:
SELECTION_1:
Channnel: Security
SELECTION_2:
EventID: 4625
SELECTION_3:
LogonType: 3
FILTER_1:
SubStatus: "0xc0000064" #Non-existent user
FILTER_2:
SubStatus: "0xc000006a" #Wrong password
condition: SELECTION_1 and SELECTION_2 and SELECTION_3 and not (FILTER_1 or FILTER_2)
Please do this
detection:
selection:
Channel: Security
EventID: 4625
LogonType: 3
filter:
- SubStatus: "0xc0000064" #Non-existent user
- SubStatus: "0xc000006a" #Wrong password
condition: selection and not filter
- When you need multiple sections, please name the first section with channel and event ID information in the
section_basic
section and other selections with meaningful names aftersection_
andfilter_
. Also, please write comments to explain anything difficult to understand. For example:
Instead of
detection:
Takoyaki:
Channel: Security
EventID: 4648
Naruto:
TargetUserName|endswith: "$"
IpAddress: "-"
Sushi:
SubjectUserName|endswith: "$"
TargetUserName|endswith: "$"
TargetInfo|endswith: "$"
Godzilla:
SubjectUserName|endswith: "$"
Ninja:
TargetUserName|re: "(DWM|UMFD)-([0-9]|1[0-2])$"
IpAddress: "-"
Daisuki:
- ProcessName|endswith: "powershell.exe"
- ProcessName|endswith: "WMIC.exe"
condition: Takoyaki and Daisuki and not (Naruto and not Godzilla) and not Ninja and not Sushi
Please do this
detection:
selection_basic:
Channel: Security
EventID: 4648
selection_TargetUserIsComputerAccount:
TargetUserName|endswith: "$"
IpAddress: "-"
filter_UsersAndTargetServerAreComputerAccounts: #Filter system noise
SubjectUserName|endswith: "$"
TargetUserName|endswith: "$"
TargetInfo|endswith: "$"
filter_SubjectUserIsComputerAccount:
SubjectUserName|endswith: "$"
filter_SystemAccounts:
TargetUserName|re: "(DWM|UMFD)-([0-9]|1[0-2])$" #Filter out default Desktop Windows Manager and User Mode Driver Framework accounts
IpAddress: "-" #Don't filter if the IP address is remote to catch attackers who created backdoor accounts that look like DWM-12, etc..
selection_SuspiciousProcess:
- ProcessName|endswith: "powershell.exe"
- ProcessName|endswith: "WMIC.exe"
condition: selection_basic and selection_SuspiciousProcess and not (selection_TargetUserIsComputerAccount
and not filter_SubjectUserIsComputerAccount) and not filter_SystemAccounts and not filter_UsersAndTargetServerAreComputerAccounts
Converting Sigma rules to Hayabusa format
We have created a backend to convert rules from Sigma to Hayabusa-compatible format here.
You can recieve the latest news about Hayabusa, rule updates, other Yamato Security tools, etc... by following us on Twitter at @SecurityYamato.