Awesome
PaSh: Light-touch Data-Parallel Shell Processing
A system for parallelizing POSIX shell scripts. Hosted by the Linux Foundation.
Service | Main | Develop |
---|---|---|
Tests | ||
Build | ||
Pages |
Quick Jump: Running PaSh | Installation | Testing | Repo Structure | Community & More | Citing
Running PaSh
To parallelize, say, ./evaluation/intro/hello-world.sh
with parallelization degree of 2× run:
./pa.sh ./evaluation/intro/hello-world.sh
Run ./pa.sh --help
to get more information about the available commands.
Jump to docs/tutorial for a longer tutorial.
Installation
On Ubuntu, Fedora, and Debian run the following to set up PaSh.
wget https://raw.githubusercontent.com/binpash/pash/main/scripts/up.sh
sh up.sh
export PASH_TOP="$PWD/pash/"
## Run PaSh with echo hi
"$PASH_TOP/pa.sh" -c "echo hi"
For more details, manual installation, or other platforms see installation instructions.
Repo Structure
This repo hosts the core pash
development. The structure is as follows:
- compiler: Shell-dataflow translations and associated parallelization transformations.
- docs: Design documents, tutorials, installation instructions, etc.
- evaluation: Shell pipelines and example scripts used for the evaluation.
- runtime: Runtime component — e.g.,
eager
,split
, and associated combiners. - scripts: Scripts related to continuous integration, deployment, and testing.
Community & More
Chat:
Mailing Lists:
- pash-devs: Join this mailing list for discussing all things
pash
- pash-commits: Join this mailing list for commit notifications
Development/contributions:
- Contribution guide: docs/contributing
- Continuous Integration Server: ci.binpa.sh
Citing
If you used PaSh, consider citing the following paper:
@inproceedings{pash2021eurosys,
author = {Vasilakis, Nikos and Kallas, Konstantinos and Mamouras, Konstantinos and Benetopoulos, Achilles and Cvetkovi\'{c}, Lazar},
title = {PaSh: Light-Touch Data-Parallel Shell Processing},
year = {2021},
isbn = {9781450383349},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3447786.3456228},
doi = {10.1145/3447786.3456228},
pages = {49–66},
numpages = {18},
keywords = {POSIX, Unix, pipelines, automatic parallelization, source-to-source compiler, shell},
location = {Online Event, United Kingdom},
series = {EuroSys '21}
}