Awesome

Overview

This is a collection of miscellaneous utilities I've written over the years. Their motivation/scope spans from nearly trivial access checking of okpaths to nascent attempts or atoms of Big Things like map-reduce in shell or extreme value estimation for benchmark time cleaning. All are tools I use in one way or another with some regularity which I thought others might find useful. There is a general vibe similar to util-linux or moreutils. A great many are Unix only tools, but some may be more cross-platform.

File Typology (What/Whether)

ft - file typer {i-node type, not file(1)/libmagic(3) type}

only - file(1)/libmagic tool to emit files whose types match

fkindc - file(1)/libmagic tool to histogram file types

notIn - Helper to manage semi-mirrored file trees

catz - Generalize zcat to many encodings, not merely gzip

Space Management (How Much/Where)

dfr - d)isk fr)ee; df with color coding & modern units

lncs - analyze a file tree for hard link structure

du - Slight improvement on GNU du

rr - Mostly a short alias for rm -rf but also faster

dups - Fast finder of exact duplicate files

fpr - File Pages Resident reporting utility like fincore

File Time Related (When)

cbtm - Back up & restore new Linux b-time stamps (creation/birth)

dirt - Recursively set dir time stamp to oldest of members

fage - file age according to various timestamps/rules

newest - b-time supporting `find -printf|sort|tail`

since - b-time supporting `find -Xnewer`

saft - SAve&restore File Times across a command operating on them

tmath - Convert/do arithmetic directly upon date & time formats

mk1 - Very fast make for many 1-to-1 input-output mappings

Benchmarking Related Utilities (How Long)

memlat - measure memory latency at various size scales

fread - Like cat but just read data (no writes)

ru - Resource Usage measurement { high-res/nicer time(1) }

etr - e)stimate t)ime r)emaining using subcommands for %done

eve - Extreme Value Estimator (e.g. true max|min of an infinite sample)

tim - Uncertain time comparison via repeated sampling & eve

edplot - Generate EDF & its confidence bands files & plot scripts

keydowns - Assess string complexity in terms of a human cost

Row/Column/Text Table Things

align - align text with better ergonomics than BSD column

flow - flow text lines into as many columns as fit with aligned output

tails - Generalizes head & tail into one with all-but compliments

cols - extract just some columns from a text file/stream

rp - A row processor program-generator maybe replacement for AWK

crp - C row processor program-generator port of rp

colSort - Sort within the columns of rows

cstats - preserve Context/Compute Column stats filter

Freeform Text Things

noc - stdin-out filter to strip ANSI CSI/OSC/SGR color escape seqs

tw - terminal-width clip/cropper with m-row bounding capability

cfold - Context folding (like csplit but to wrap lines)

unfold - Oft neglected inverse-to-wrapping/folding process

ww - Dynamic programming based word wrapper

widths - Compute & emit line widths/lengths | distro

jointr - join strace "unfinished ..." with conclusion

ndelta - Numerical difference between two reports utility

tmpls - A fast string template interpolater

Pipeline Calculation Tools

topn - Fast streaming 1-pass top-N over M columns

oft - Approximately most often items via a low-memory algorithm

uce - Unique/distinct Count Estimate via a low-memory algorithm

System Administration on Unix/POSIX/Linux

holes - count/map hole & data segments of files

fsids - file system user & group id histogram

chom - Enforce group owner & segregated perms in file trees

thermctl - Thermal Control for before CPU makers thermally throttled

pid2 - Wrap Linux process PID table to first past target

sr - System Request Key - rapidly act on Linux systems

Tty Handling

tattr - Terminal attribute access (like cligen/humanUt)

wsz - Report terminal size in cells, pixels, and cell size

Miscellaneous/Islands Unto Themselves

noa - "--"-aware Python-like indexing of non-option arguments

okpaths - Validate/trim PATH-like vars by probing the system

nrel - Edit .nimble version, commit, tag, push & maybe release

dirq - Kind of its own system-building atom thing

funnel - A reliable, record boundary respecting "FIFO funnel"

stripe - Run jobs in parallel w/slot key vars/seqNos/shell elision

bu/rs- Reservoir Subset/Sampler Of Lines In A File/[T] library

wgt - Weighted random sampler with fancy weighting

zipf - Random samples according to Zipf distribution

niom - nio moments w/hard dep on adix efficient histogram/quantiles

ac - aped commands with aping rules stored in local file tree

Meta-Commentary

Every tool has a help message hopefully useful enough to be mostly autonomous or at least a good reminder of its own README in doc/TOOLNAME.md. I should write some man pages starting with help2man, but AFAICT, nimble, will not install them anyway (there's no mention of the whole word "man" on any issue, PR, or anywhere in the VC history of nimble, nor any default target one could add to MANPATH | /etc/man*.conf). Many doc/TOOLNAME.md's should grow more complete/better examples. I tried to keep the source code scrutable, but also to the point. Doc PRs are always welcome. Though we all try, there is no substitute for initial confusion resolving at improving docs). Some packaging meta-commentary is here.