Awesome
GHC whole program compiler project
The project consists of GHC wpc-plugin and the corresponding External STG IR and tooling.
The wpc-plugin is a compiler plugin for GHC 9.6 or newer. It exports the STG IR (.modpak)
for the compiled modules and linker metadata (.ghc_stgapp
) at application link time.
Presentation video
- Why a GHC Whole Program Compiler Mode Would Be Useful (slides)
- Why and How the External STG Interpreter is Useful (slides) (demo code)
Readings
- Introducing GHC whole program compiler (GHC-WPC)
- GHC-WPC thread on ghc-devs mailing list
- External STG Interpreter
External STG tools (Ext-STG)
gen-exe
main compiler driver, it produces executable from.ghc_stgapp
files.gen-obj
compiles STG IR files.o_stgbin
to object code.o
. (gen-exe calls it)ext-stg
CLI tool for external STG IR, it can pretty print.o_stgbin
files.
Why?
- to make it easy to develop new backends for GHC without extending Cabal with new targets
- to facilitate compiler/PL research that needs real world programs to analyse
- to allow whole program analysis (new insights might be adopted to incremental compilers)
- to escape from GHC codebase to the mainstream Haskell UX/DX that allows to use any library
- to allow program observation with arbitrary precision
- to make it easy to focus on the compiler backend development without hacking GHC
- to allow other compilers to target GHC/STG and the feature rich RTS
Build
external stg tooling
stack install
wpc-plugin
- Install zip-cmd, a simple CLI for the
zip
packagecabal install zip-cmd
- Compile the
wpc-plugin
Thewpc-plugin
has a speparatestack.yaml
because it uses the plugin API ofGHC 9.6.1
.cd wpc-plugin stack build
- Find the built
libwpc-plugin.[so|dylib|dll]
ln -s `find . -type f -name 'libwpc-plugin.so' -o -name 'libwpc-plugin.dylib' -o -name 'libwpc-plugin.dll' | head -1`
Usage
It is required to use GHC 9.6.1.
cabal
Add the following lines to your project's cabal.project
:
package *
ghc-options:
-fplugin-trustworthy
-fplugin-library=PATH_TO/libwpc-plugin.so;wpc-plugin-unit;WPC.Plugin;[]
stack
Add the following lines to your project's stack.yaml
:
apply-ghc-options: everything
ghc-options:
"$everything":
-fplugin-trustworthy
-fplugin-library=PATH_TO/libwpc-plugin.so;wpc-plugin-unit;WPC.Plugin;[]
TODO
Ext-STG IR
- export IdInfo (without it
gen-exe
compiles -O0 executables)
UnZip with Zstd support
The .modpak
and .fullpak
files use Zstd compression method that was introduced in the Zip 6.3.8 standard in 2020.
The GHC-WPC tooling can handle Zstd zip files out of the box.
But if you'd like to unpack the .modpak
and .fullpak
files manually then you'll need an unzip
version with Zstd support.
https://github.com/csabahruska/unzip-zstd