Awesome

Sheer Heart Attack

sheer-heart-attack is a debugging tool that can execute any command on process/host metrics trigger.

screencast

Features

Easy to use (just execute sheer-heart-attack launch).
Track process and/or host metrics. and execute specified command when threshold is exceeded.
Record the STDOUT and STDERR of the executed command in the structured log.
Slack notification.

Quick Start

This is the recommended usage.

root@kilr_q:~# source <(curl -sL https://raw.githubusercontent.com/k1LoW/sheer-heart-attack/main/use)
You can use `sheer-heart-attack` command in this session.
root@kilr_q:~# sheer-heart-attack launch

In the case of fish :fish:

root@kilr_q:~# curl -sL https://git.io/sheer-heart-attack-fish | source
You can use `sheer-heart-attack` command in this session.
root@kilr_q:~# sheer-heart-attack launch

Install

manually:

Download binany from releases page

go get:

go get github.com/k1LoW/sheer-heart-attack

Usage

Just execute sheer-heart-attack launch.

Launch Options (Flags):

Option	Default	Purpose
`pid`		PID of the process.
`threshold`	`cpu > 5 \|\| mem > 10`	Threshold conditions.
`interval`	`5s`	Interval of checking if the threshold exceeded.
`attempts`	`1`	Maximum number of attempts continuously exceeding the threshold.
`command`		Command to execute when the maximum number of attempts is exceeded.
`times`	`1`	Maximum number of command executions. If times < 1, track and execute until timeout.
`timeout`	`1day`	Timeout of tracking.
`slack-channel`		Slack channel to notify.
`slack-mention`		Slack mention. (`@here` or user_id `@UXXXXXXXXX`)

Set `threshold` using operators

The following operators can be used to set the threshold:

+, -, *, /, ==, !=, <, >, <=, >=, not, and, or, !, &&, ||

For example, you can set the threshold as follows

cpu > 10 and mem > 20
(user + system) > 50 || iowait > 50
load1 > 5 or load15 > 2

Slack Notification

sheer-heart-attack find and use Slack Incomming Webhook URL via envirionment variables ( SLACK_INCOMMING_WEBHOOK_URL, SLACK_WEBHOOK_URL, SLACK_URL )

slack

Support Metrics

Metric
`proc_cpu`	Percentage of the CPU time the process uses (percent).
`proc_mem`	Percentage of the total RAM the process uses (percent).
`proc_rss`	Non-swapped physical memory the process uses (bytes).
`proc_vms`	Amount of virtual memory the process uses (bytes).
`proc_swap`	Amount of memory that has been swapped out to disk the process uses (bytes).
`proc_open_files`	Amount of files and file discripters opend by the process. linux only
`cpu`	Percentage of cpu used.
`mem`	Percentage of RAM used.
`swap`	Amount of memory that has been swapped out to disk (bytes).
`user`	Percentage of CPU utilization that occurred while executing at the user level.
`system`	Percentage of CPU utilization that occurred while executing at the system level.
`idle`	Percentage of time that CPUs were idle and the system did not have an outstanding disk I/O request.
`nice`	Percentage of CPU utilization that occurred while executing at the user level with nice priority.
`iowait`	Percentage of time that CPUs were idle during which the system had an outstanding disk I/O request.
`irq`	Percentage of time spent by CPUs to service hardware interrupts.
`softirq`	Percentage of time spent by CPUs to service software interrupts.
`steal`	Percentage of time spent in involuntary wait by the virtual CPUs while the hypervisor was servicing another virtual processor.
`guest`	Percentage of time spent by CPUs to run a virtual processor.
`guest_nice`	Percentage of time spent by CPUs to run a virtual processor with nice priority.
`load1`	Load avarage for 1 minute.
`load5`	Load avarage for 5 minutes.
`load15`	Load avarage for 15 minutes.

Alternatives

ProcDump for Linux - A Linux version of the ProcDump Sysinternals tool