Awesome

repren

✨ NEW: v1.0 is now updated for Python 3.10-3.13! ✨

Rename Anything

Repren is a simple but flexible command-line tool for rewriting file contents according to a set of regular expression patterns, and to rename or move files according to patterns. Essentially, it is a general-purpose, brute-force text file refactoring tool.

For example, repren could rename all occurrences of certain class and variable names in a set of Java source files, while simultaneously renaming the Java files according to the same pattern.

It's more powerful than usual options like perl -pie, rpl, or sed:

• It can also rename files, including moving files and creating directories.

• It supports fully expressive regular expression substitutions.

• It performs group renamings, i.e. rename "foo" as "bar", and "bar" as "foo" at once, without requiring a temporary intermediate rename.

• It is careful. It has a nondestructive mode, and prints clear stats on its changes. It leaves backups. File operations are done atomically, so interruptions never leave a previously existing file truncated or partly edited.

• It supports "magic" case-preserving renames that let you find and rename identifiers with case variants (lowerCamel, UpperCamel, lower_underscore, and UPPER_UNDERSCORE) consistently.

• It has this nice documentation!

If file paths are provided, repren replaces those files in place, leaving a backup with extension ".orig".

If directory paths are provided, it applies replacements recursively to all files in the supplied paths that are not in the exclude pattern. If no arguments are supplied, it reads from stdin and writes to stdout.

Examples

Patterns can be supplied in a text file, with one or more replacements consisting of regular expression and replacement. For example:

# Sample pattern file
frobinator<tab>glurp
WhizzleStick<tab>AcmeExtrudedPlasticFunProvider
figure ([0-9+])<tab>Figure \1

(Where <tab> is an actual tab character.) Each line is a replacement. Empty lines and #-prefixed comments are ignored.

As a short-cut, a single replacement can be specified on the command line using --from (match) and --to (replacement).

Examples:

# Here `patfile` is a patterns file.
# Rewrite stdin:
repren -p patfile < input > output

# Shortcut with a single pattern replacement (replace foo with bar):
repren --from foo --to bar < input > output

# Rewrite a few files in place, also requiring matches be on word breaks:
repren -p patfile --word-breaks myfile1 myfile2 myfile3

# Rewrite whole directory trees. Since this is a big operation, we use
# `-n` to do a dry run that only prints what would be done:
repren -n -p patfile --word-breaks --full mydir1

# Now actually do it:
repren -p patfile --word-breaks --full mydir1

# Same as above, for all case variants:
repren -p patfile --word-breaks --preserve-case --full mydir1

# Same as above but including only .py files and excluding the tests directory
# and any files or directories starting with test_:
repren -p patfile --word-breaks --preserve-case --full --include='.*[.]py$' --exclude='tests|test_.*' mydir1

Usage

Run repren --help for full usage and flags.

If file paths are provided, repren replaces those files in place, leaving a backup with extension ".orig". If directory paths are provided, it applies replacements recursively to all files in the supplied paths that are not in the exclude pattern. If no arguments are supplied, it reads from stdin and writes to stdout.

Alternatives

Aren't there standard tools for this already?

It's a bit surprising, but not really. Getting the features right is a bit tricky, I guess. The standard answers like sed, perl, awk, rename, Vim macros, or even IDE refactoring tools, often cover specific cases, but tend to be error-prone or not offer specific features you probably want. Things like nondestructive mode, file renaming as well as search/replace, multiple simultaneous renames/swaps, or renaming enclosing parent directories. Also many of these vary by platform, which adds to the corner cases. Inevitably you end up digging through the darker corners of some man page or writing ugly scripts that would scare your mother.

Installation

No dependencies except Python 3.10+. It's easiest to install with pip:

pip install repren

Or, since it's just one file, you can copy the repren.py script somewhere convenient and make it executable.

Try It

Let's try a simple replacement in my working directory (which has a few random source files):

$ repren --from frobinator-server --to glurp-server --full --dry-run .
Dry run: No files will be changed
Using 1 patterns:
  'frobinator-server' -> 'glurp-server'
Found 102 files in: .
- modify: ./site.yml: 1 matches
- rename: ./roles/frobinator-server/defaults/main.yml -> ./roles/glurp-server/defaults/main.yml
- rename: ./roles/frobinator-server/files/deploy-frobinator-server.sh -> ./roles/glurp-server/files/deploy-frobinator-server.sh
- rename: ./roles/frobinator-server/files/install-graphviz.sh -> ./roles/glurp-server/files/install-graphviz.sh
- rename: ./roles/frobinator-server/files/frobinator-purge-old-deployments -> ./roles/glurp-server/files/frobinator-purge-old-deployments
- rename: ./roles/frobinator-server/handlers/main.yml -> ./roles/glurp-server/handlers/main.yml
- rename: ./roles/frobinator-server/tasks/main.yml -> ./roles/glurp-server/tasks/main.yml
- rename: ./roles/frobinator-server/templates/frobinator-webservice.conf.j2 -> ./roles/glurp-server/templates/frobinator-webservice.conf.j2
- rename: ./roles/frobinator-server/templates/frobinator-webui.conf.j2 -> ./roles/glurp-server/templates/frobinator-webui.conf.j2
Read 102 files (190382 chars), found 2 matches (0 skipped due to overlaps)
Dry run: Would have changed 2 files, including 0 renames

That was a dry run, so if it looks good, it's easy to repeat that a second time, dropping the --dry-run flag. If this is in git, we'd do a git diff to verify, test, then commit it all. If we messed up, there are still .orig files present.

Patterns

Patterns can be supplied using the --from and --to syntax above, but that only works for a single pattern.

In general, you can perform multiple simultaneous replacements by putting them in a patterns file. Each line consists of a regular expression and replacement. For example:

# Sample pattern file
frobinator<tab>glurp
WhizzleStick<tab>AcmeExtrudedPlasticFunProvider
figure ([0-9+])<tab>Figure 

(Where <tab> is an actual tab character.)

Empty lines and #-prefixed comments are ignored. Capturing groups and back substitutions (such as
above) are supported.

Examples

# Here `patfile` is a patterns file.
# Rewrite stdin:
repren -p patfile < input > output

# Shortcut with a single pattern replacement (replace foo with bar):
repren --from foo --to bar < input > output

# Rewrite a few files in place, also requiring matches be on word breaks:
repren -p patfile --word-breaks myfile1 myfile2 myfile3

# Rewrite whole directory trees. Since this is a big operation, we use
# `-n` to do a dry run that only prints what would be done:
repren -n -p patfile --word-breaks --full mydir1

# Now actually do it:
repren -p patfile --word-breaks --full mydir1

# Same as above, for all case variants:
repren -p patfile --word-breaks --preserve-case --full mydir1

Notes

• As with sed, replacements are made line by line by default. Memory permitting, replacements may be done on entire files using --at-once.

• As with sed, replacement text may include backreferences to groups within the regular expression, using the usual syntax: \1, \2, etc.

• In the pattern file, both the regular expression and the replacement may contain the usual escapes \n, \t, etc. (To match a multi-line pattern, containing \n, you must must use --at-once.)

• Replacements are all matched on each input file, then all replaced, so it's possible to swap or otherwise change names in ways that would require multiple steps if done one replacement at at a time.

• If two patterns have matches that overlap, only one replacement is applied, with preference to the pattern appearing first in the patterns file.

• If one pattern is a subset of another, consider if --word-breaks will help.

• If patterns have special characters, --literal may help.

• The case-preserving option works by adding all case variants to the pattern replacements, e.g. if the pattern file has foo_bar -> xxx_yyy, the replacements fooBar -> xxxYyy, FooBar -> XxxYyy, FOO_BAR -> XXX_YYY are also made. Assumes each pattern has one casing convention.

• The same logic applies to filenames, with patterns applied to the full file path with slashes replaced and then and parent directories created as needed, e.g. my/path/to/filename can be rewritten to my/other/path/to/otherfile. (Use caution and test with -n, especially when using absolute path arguments!)

• Files are never clobbered by renames. If a target already exists, or multiple files are renamed to the same target, numeric suffixes will be added to make the files distinct (".1", ".2", etc.).

• Files are created at a temporary location, then renamed, so original files are left intact in case of unexpected errors. File permissions are preserved.

• Backups are created of all modified files, with the suffix ".orig".

• By default, recursive searching omits paths starting with ".". This may be adjusted with --exclude. Files ending in .orig are always ignored.

• Data is handled as bytes internally, allowing it to work with any encoding or binary files. File contents are not decoded unless necessary (e.g., for logging). However, patterns are specified as strings in the pattern file and command line arguments, and file paths are handled as strings for filesystem operations.

Contributing

Contributions and issues welcome! Check the output of the test script and if it has changed or needs updating, and commit the clean log changes if you submit a PR.

License

MIT.