Home

Awesome

Shrinko8

A set of Pico-8 & Picotron cart tools, with a focus on shrinking code size.

You can run it online here.

You can download a recent Windows Executable here.

Otherwise, requires Python 3.7 or above to run.

Reading/Writing PNGs additionally requires the Pillow module (python -m pip install pillow to install)

Download the latest version of the source here.

The major supported features are:

Minification

Greatly reduces the character count of your cart, as well as greatly improves its compression ratio (so that its compressed size is smaller) and can reduce the number of tokens as well.

There are command line options to choose how aggressively to minify, as well as what metric (compressed size or character count) to focus on minifying.

It's recommended to combine minification with conversion to png (as seen in the examples below), as Shrinko8 is able to compress code better and can thus fit carts into pngs that Pico-8 cannot.

To minify your p8 cart:

You have several options, depending on how much minification you need:

The simplest approach, which gives good results and works on any cart:

python shrinko8.py path-to-input.p8 path-to-output.png --minify-safe-only

You can also add --focus-tokens, --focus-chars, or --focus-compressed to the command - depending on what you want Shrinko8 to focus on reducing.

The most aggressive approach, which gives the best results, but sometimes requires you to give additional information to shrinko8 to ensure it minifies your cart correctly:

python shrinko8.py path-to-input.p8 path-to-output.png --minify

If you want to minify, but also to keep your cart easily debuggable and reasonably readable by others, you can do:

python shrinko8.py path-to-input.p8 path-to-output.png --minify-safe-only --no-minify-rename --no-minify-lines

You can also minify to a p8 file (or a lua file), e.g:

python shrinko8.py path-to-input.p8 path-to-output.p8 --minify-safe-only

Debugging the minified cart

If the minified cart errors or misbehaves, here are some tips:

Minify options

You can specify what the minification should focus on reducing via additional command-line options:

You can disable parts of the minification process via additional command-line options:

You can control how safe the minification is (see details about unsafe minifications):

Additional options:

Operation details

Pitfalls of full minification

When using --minify without --minify-safe-only, Shrinko8 makes - by default - some assumptions about your cart:

These assumptions allow Shrinko8 to - for example - freely rename identifiers used to index tables.

If these assumptions don't hold, the minified cart won't work properly, e.g:

local my_obj = {key=123} -- here, key is an identifier.
?my_obj.key -- OK. Here, key is an identifier again.
local my_key = "key" -- here, key is a string.
?my_obj[my_key] -- BUG! my_obj will not have a "key" member after minification!

In such cases, you have multiple ways to tell Shrinko8 precisely how your cart breaks these assumptions, allowing you to achieve better minification than would be possible with just --minify-safe-only:

Renaming specific strings

You can add a --[[member]] comment before a string to have the minifier rename it as if it were an identifier.

E.g:

local my_key = --[[member]]"key" -- here, key is a string but is renamed as if it were an identifier
local my_obj = {key=123} -- here, key is an identifier
?my_obj[my_key] -- success, this prints 123 even after minification

You can also use this with multiple keys split by comma (or any other characters):

local my_keys = split --[[member]]"key1,key2,key3" -- here, each of key1, key2 and key3 is renamed

And you can similarly use --[[global]] for globals:

local my_key = --[[global]]"glob"
glob = 123
?_ENV[my_key] -- 123

Advanced: if you have string literals in some special format that you're parsing into a table (like "key=val,key2=val2"), you can use this custom python script - or a variant thereof - to allow only the keys within the string to be renamed.

Preserving identifiers across the entire cart

You can instruct the minifier to preserve certain identifiers across the entire cart by adding a --preserve: comment anywhere in the code:

--preserve: my_global_1, my_global_2, update_*, *.my_member, my_env.*

If you prefer, you can instead pass this information in the command line, e,g:

python shrinko8.py path-to-input.p8 path-to-output.png --minify --preserve "my_global_1, my_global_2, update_*, *.my_member, my_env.*"

You can combine wildcards and negation (!) to preserve everything except some identifiers:

--preserve: *.*, !*.my_*

Renaming table keys the same way as globals

You can instruct the minifier to rename table keys the same way as globals (allowing you to freely mix _ENV and other tables), by adding the following comment in the code:

--preserve: *=*.*

If you prefer, you can instead pass --preserve "*=*.*" to the command line.

Controlling renaming of all keys of a table

You can use --[[preserve-keys]], --[[global-keys]] and --[[member-keys]] to affect how all keys of a table are renamed.

This can be applied on either table constructors (aka {...}) or on variables. When applying on variables, the hint affects all members accessed through that variable, as well as any table constructors directly assigned to it.

local --[[preserve-keys]]my_table = {preserved1=1, preserved2=2}
my_table.preserved1 += 1 -- all member accesses through my_table are preserved
?my_table["preserved1"]

-- here, {preserved3=3} is not directly assigned to my_table and so needs its own hint
my_table = setmetatable(--[[preserve-keys]]{preserved3=3}, my_meta)
?my_table["preserved3"]

-- while assigning directly to _ENV no longer requires a hint, indirect assignment like below does:
local env = --[[global-keys]]{assert=assert, add=add}
do
  local _ENV = env
  assert(add({}, 1) == 1)
end

This can be also be useful when assigning regular tables to _ENV:

-- hints on an _ENV local affects all globals in its scope
for --[[member-keys]]_ENV in all({{x=1,y=5}, {x=2,y=6}}) do
  x += y + y*x
end

Advanced - Controlling renaming of specific identifier occurrences

The --[[global]], --[[member]] and --[[preserve]] hints can also be used on a specific occurrence of an identifier to change the way it's renamed.

Usually, there are easier ways to control renaming (such as by preserving identifiers across the entire cart or controlling renaming of all keys in a table), but this option is here for cases where you need precise control over how to rename each occurence.

do
  -- NOTE: can be more easily achieved via --[[global-keys]]
  local _ENV = {--[[global]]assert=assert}
  assert(true)
end
-- NOTE: can be more easily achieved via --[[member-keys]]
for _ENV in all({{x=1}, {x=2}}) do
  --[[member]]x += 1
end

Advanced - Renaming Built-in Pico-8 functions

For cases like tweet-carts, when you use a builtin function multiple times throughout your cart, you often want to assign it to a shorter name at the beginning of the cart. With shrinko8, you can keep using the full name of the builtin, but tell the minifer to only preserve the builtin when it's first accessed, as follows:

--preserve: !circfill, !rectfill
circfill, rectfill = --[[preserve]]circfill, --[[preserve]]rectfill
circfill(10,10,20); circfill(90,90,30)
rectfill(0,0,100,100); rectfill(20,20,40,40)

Above, all uses of circfill and rectfill are renamed except for the ones preceded by --[[preserve]]

Be aware that doing this won't reduce the compressed size of the cart, and will increases the token count (due to the assignment), so it's only for when you care about character count above all else.

Advanced - Explicit renaming

While Shrinko8 has good heuristics for choosing identifier names, it's still possible to improve upon them when hand-minifying carts (useful especially when trying to fit small carts under some chosen limit).

In order to still be able to use Shrinko8 in such cases, a hint is provided to instruct Shrinko8 how to rename specific variables:

function --[[rename::f]]func(--[[rename::a]]arg)
    local --[[rename::b]]val = arg
end

A rename hint affects all instances of the marked variable.

Prevent merging of specific statements

You can insert --[[no-merge]] between two statements to ensure they're not merged, e.g:

-- note: this example requires --focus-tokens to see the effect
local weird_table = setmetatable({add_me=0}, {
    __newindex=function(tbl, key, val) rawset(tbl, key, val + t.add_me) end
})
-- the following statements do not do the same thing if combined into one
-- aka: weird_table.add_me, weird_table.new_key = 3, 4
-- so we can add --[[no-merge]] between them to ensure they're not merged.
weird_table.add_me = 3
--[[no-merge]]
weird_table.new_key = 4

Keeping comments

You can keep specific comments in the output via:

--keep: This is a comment to keep
-- But this comment is gone after minify

Constants

During minification, Shrinko8 will automatically replace most constant expressions with their value:

func(60*60)
-- becomes:
func(3600)

func('the answer is: '..1+3*2)
-- becomes:
func('the answer is: 7')

In addition, variables that are declared with the --[[const]] hint are treated as constants:

--[[const]] k_hero_spr = 4
spr(k_hero_spr, x, y)
-- becomes:
spr(4, x, y)

--[[const]] version = 'v1.2'
?'version: '..version
-- becomes:
?'version: v1.2'

-- the --[[const]] hint can apply to either individual variables or entire local statements
--[[const]] local k_rock,k_box,k_wall = 4,5,6
objs={k_rock,k_wall,k_wall,k_box}
-- becomes:
objs={4,6,6,5}

-- some builtin functions can be used inside const declarations
--[[const]] k_value = 2.5
--[[const]] derived = flr(mid(k_value, 1, 5))
?derived
-- becomes:
?2

Furthermore, constant if and elseif branches are removed appropriately, allowing you to easily keep debug code in your source files, enabling it by simply changing the value of a variable:

--[[const]] TRACE = false
--[[const]] DEBUG = true

if (TRACE) ?"something happened!"
if DEBUG then
  spr(debug_spr, 10, 10)
end

-- becomes:
spr(debug_spr,10,10)

Some details to keep in mind:

Passing constants via command line

You can even declare constants in the command line, if you prefer:

python shrinko8.py path-to-input.p8 path-to-output.p8 --minify-safe-only --const DEBUG true --const SPEED 2.5 --str-const VERSION v1.2

--[[CONST]] SPEED = 0.5 -- default value
if DEBUG then
  ?'debug version ' .. (VERSION or '???')
end
hero = 0
function _update()
  hero += SPEED
end

Becomes: (disregarding other minifications)

?"debug version v1.2"
hero = 0
function _update()
  hero += 2.5
end

Limitations

Keep in mind that in some cases, Shrinko8 will play it safe and avoid a computation whose result is questionable or has a high potential to change between pico8 versions. If this prevents a --[[const]] variable from being assigned a constant, Shrinko8 will warn about this:

-- here, abs overflows (due to receiving -0x8000), and shrinko8 chooses not to rely on the overflow behaviour
--[[const]] x = abs(0x7fff+1)-1
?x

-- warning:
--tmp.lua:1:13: Local 'x' is marked as const but its value cannot be determined due to 'abs(0x7fff+1)'

-- Becomes only:
x=abs(32768)-1
?x

If you find such limitations that you'd like to see lifted, feel free to open an issue.

Finally, note that:

Linting

Linting finds common code issues in your cart, like forgetting to use a 'local' statement

To lint your p8 cart:

python shrinko8.py path-to-input.p8 --lint

You can combine linting with other operations:

python shrinko8.py path-to-input.p8 path-to-output.p8 --lint --count --minify

Linting options

You can disable certain lints globally via additional command-line options:

Normally, a lint failure prevents cart creation, but --no-lint-fail overrides that.

Normally, lint errors are displayed in a format useful for external editors, showing the line number in the whole .p8 file. However, you can use --error-format tabbed to show the pico8 tab number and line number inside that tab instead.

Misc. options:

Undefined variable lints

In Pico-8 (and lua in general), variables that aren't explicitly declared as local (via a local statement) are implicitly global. This can cause all sorts of bugs and headaches if you typo the name of a local or forget to declare a local.

This lint alerts you when you're accessing a variable that wasn't declared as local and isn't a known global variable, e.g:

function f()
    x, y = 10, 20 -- lint warning: you probably meant to use 'local' here instead of assigning to global variables.
    while x < y do stuff(x, y) end
end

Defining global variables

The linter normally allows you to define global variables in the global scope or in the _init function. If you don't, your options are either:

Tell the linter about the globals it didn't see you define via the --lint: hint:

--lint: global_1, global_2
function f()
    dostuff(global_1, global_2)
end

Tell the linter to allow you to define globals (by assigning to them) in a specific function via the --lint: func::_init hint:

--lint: func::_init
function my_init()
    global_1, global_2 = 1, 2 -- these globals can be used anywhere since they're assigned here
end

Re-assigning built-in globals

Similarly, to protect against accidental use of built-in globals like run or t, the linter only allows you to assign to built-in globals in the global scope or in an _init function:

function f()
    t = func() -- lint warning: you probably meant to use 'local' here, even though t is a built-in global
end

If you do want to reassign some built-in global anywhere, you can use --lint:

--lint: print
function f()
    local old_print = print
    print = function() end
    call_something()
    print = old_print
end

Unused variable lints

This lint alerts you when you've declared a variable but never used it, which is usually a mistake.

It also tells you when the last parameter of a function is unused, as that's either a mistake or a waste of a token.

To tell the linter that some specific local is OK to be unused, named it beginning with underscore (e.g. _ or _some_name). E.g:

do
  local _, _, x, y = get_stuff() -- lint warning about y (but not about _) - you probably meant to pass it to do_stuff
  do_stuff(x, x)
end

If you have false positives in your cart due to globals being used via _ENV, you can disable this check just for globals via --no-lint-unused-global.

Another option is to use the --lint: used::<var> hint:

--lint: used::global_1, used::global_2
function global_1() end
global_2 = ""

Duplicate variable lints

This lint alerts you when you declare a local with the same name as a local in a parent scope (even across functions).

This can cause confusion and bugs since you can accidentally use the wrong local. E.g:

function f()
  for i=1,10 do
    do_stuff(i)
    for i=1,5 do -- lint warning about i
      do_more(i)
    end
  end
end

The linter allows duplicate variables if they're all named _:

local _, _, x, y, _, z = stuff()

It also alerts you when you declare a local with the same name as a global you defined or used elsewhere in your cart, which is similarly confusing, E.g.:

function maths(arg)
    return sin(arg) + cos(arg)
end
function confess(sin) -- lint warning about sin
    do_stuff(sin)
    -- ...
    show_some_ui(sin(3)) -- oops!
end

Getting Cart Size

You can enable printing the number of tokens, characters, and compressed bytes used by the code in the cart (including percentages):

python shrinko8.py path-to-input.p8 --count

E.g may print:

tokens: 8053 98%
chars: 30320 46%
compressed: 12176 77%

Note that the compressed size is how this tool would compress the code, which is better than how Pico-8 would.

You can combine counting with other operations, in which case the counts are of the output cart, not the input cart:

python shrinko8.py path-to-input.p8 path-to-output.p8 --lint --count --minify

In such cases, you can also use --input-count to count the number of tokens, characters, and compressed bytes (if applicable) of the input cart.

If you're not interested in the number of tokens or in the compressed size, you can use --no-count-tokenize or --no-count-compress to avoid tokenizing or compressing the cart just to get the count. (You will still see the count if the tokenize/compress had to be done anyway, though)

Format Conversion

Shrinko8 supports multiple cart formats, and allows converting between them:

E.g:

python shrinko8.py path-to-input.p8 path-to-output.png
python shrinko8.py path-to-input.png path-to-output.rom
python shrinko8.py path-to-input.rom path-to-output.lua
python shrinko8.py path-to-export/windows/data.pod path-to-output.p8

By default, the format is determined by the file extension, but you can specify it explicitly via:

You can combine conversion with other operations:

python shrinko8.py path-to-input.p8 path-to-output.rom --count --lint --minify

Specifying the format is also useful when using the standard input/output (via -), e.g.:

python shrinko8.py path-to-input.p8 - --minify --format lua (This prints minified lua to stdout)

You can convert a cart to multiple formats at once using --extra-output path [format]:

python shrinko8.py path-to-input.p8 path-to-output.png --extra-output path-to-output.p8 --extra-output path-to-output.rom

You can additionally export the cart's spritesheet and label:

python shrinko8.py path-to-input.p8 path-to-output.png --extra-output path-to-spritesheet.png spritesheet --extra-output path-to-label.png label

Specifying custom labels & titles

Normally, shrinko8 will take the label and title (if any) from the input cart, same as pico8 does.

However, it is also possible to override the label from a custom 128x128 screenshot via --label <path> and the title via --title "some title"

Merging multiple carts into one

You can tell Shrinko8 to merge specific sections from other carts into the input cart using --merge path sections [format].

The following example takes the label from label-cart.p8 and sfx & music from sounds-cart.p8:

python shrinko8.py path-to-input.p8 path-to-output.png --merge label-cart.p8 label --merge sounds-cart.p8 sfx,music

The following example imports the spritesheet from a 128x128 image at spritesheet.png

python shrinko8.py path-to-input.p8 path-to-output.png --merge spritesheet.png gfx spritesheet

Reading and writing exported formats

Shrinko8 supports reading and writing exported formats. Creating exports through Shrinko8 can be useful in cases when Pico8's compression algorithm isn't able to fit your cart into the export, while Shrinko8's can.

Creating an export requires you to have a copy of Pico8 and provide the pico8.dat file that comes with it as an argument to Shrinko8, as seen below.

Reading exports

Shrinko8 can read the following exports:

When you pass an export as the input parameter to Shrinko8, it will - by default - read the main cart inside.

If the export contains more than one cart, you can use:

Creating exports

Shrinko8 can create the following exports:

When you pass an export as the output parameter to Shrinko8, it will - by default - try to create a new export containing a single cart.

However, for that to work, you need to also supply --pico8-dat <path to pico8.dat file inside your pico8 directory> to Shrinko8, e.g:

python shrinko8.py path-to-input.p8 path-to-output.bin --pico8-dat c:/pico8/pico8.dat

You can create a multi-cart export by supplying additional input carts:

python shrinko8.py path-to-main-cart.p8 extra-cart-1.p8 extra-cart-2.p8 path-to-output.bin --pico8-dat c:/pico8/pico8.dat

If you need to explicitly specify the type of each additional input cart, you can instead use --extra-input

Also, if both the input and output are exports, all carts from the input get placed in the output, unless --cart is explicitly specified.

Unminification

You can undo some of the effects of minification, or just reformat the cart's code in a consistent manner:

python shrinko8.py path-to-input.p8 path-to-output.p8 --unminify

Of course, renaming cannot be undone, so the resulting code may still not be readable.

Options:

Custom Python Script

For advanced usecases, you can create a python script that will be called to preprocess or postprocess the cart before/after the other steps.

This can be used for:

To run, use --script <path>, here shown together with other tools:

python shrinko8.py path-to-input.p8 path-to-output.png --count --lint --minify --script path-to-script.py

You can also pass arguments to your script by putting them after --script-args:

python shrinko8.py path-to-input.p8 path-to-output.png --count --lint --minify --script path-to-script.py --script-args my-script-arg --my-script-opt 123

Example python script showing the API and various capabilities:

# this is called after your cart is read but before any processing is done on it:
def preprocess_main(cart, args, **_):
    print("hello from preprocess_main!")

    # 'cart' contains 'code' and 'rom' attributes that can be used to read or modify it
    # 'cart.code' is a pico8 string where each char is between '\0' and '\xff'
    #             use to/from_p8str in pico_defs.py to convert a pico8 string from/to a unicode string
    #             use decode/encode_p8str in pico_defs.py to convert a pico8 string from/to raw bytes
    # 'cart.rom' is a bytearray with some extra APIs like get16/set32/etc (see Memory in pico_defs.py)

    # copy the spritesheet from another cart
    from pico_cart import read_cart
    other_cart = read_cart("test.p8") # can be used to read p8 or png carts
    cart.rom[0x0000:0x2000] = other_cart.rom[0x0000:0x2000]

    # encode binary data into a string in our cart
    # our cart's code should contain a string like so: "$$DATA$$"
    from pico_utils import bytes_to_string_contents
    with open("binary.dat", "rb") as f:
        cart.code = cart.code.replace("$$DATA$$", bytes_to_string_contents(f.read()))

    # args.script_args contains any arguments sent to this script
    import argparse
    parser = argparse.ArgumentParser()
    parser.add_argument("arg", help="first arg sent to script", nargs="?")
    parser.add_argument("--my-script-opt", type=int, help="option sent to script")
    opts = parser.parse_args(args.script_args)
    print("Received args:", opts.arg, opts.my_script_opt)

# this is called before your cart is written, after it was fully processed
def postprocess_main(cart, **_):
    print("hello from postprocess_main!")

    # dump the code of the cart to be written
    from pico_defs import from_p8str
    with open("out.txt", "w", encoding="utf8") as f:
        f.write(from_p8str(cart.code)) # from_p8str converts the code to unicode

    # write an extra cart based on the current cart, but with a zeroed spritesheet, in both p8 and png formats
    from pico_cart import write_cart, CartFormat
    new_cart = cart.copy()
    new_cart.rom[0x0000:0x2000] = bytearray(0x2000) # zero it out
    write_cart("new_cart.p8", new_cart, CartFormat.p8)
    write_cart("new_cart.p8.png", new_cart, CartFormat.png)

    # write a new cart with the same rom but custom code, in rom format
    from pico_cart import Cart, CartFormat, write_cart
    from pico_defs import to_p8str
    new_cart = Cart(code=to_p8str("-- rom-only cart 🐱"), rom=cart.rom)
    write_cart("new_cart2.rom", new_cart, CartFormat.rom)

Advanced - custom sub-language

For really advanced usecases, if you're embedding a custom language inside the strings of your pico-8 code, you can let Shrinko8 know how to lint & minify it.

E.g. this allows renaming identifiers shared by both the pico-8 code and the custom language.

Mark the language with --[[language::<name>]] in the code:

eval(--[[language::evally]][[
    circfill 50 50 20 7
    my_global_var <- pack
    rawset my_global_var .some_member 123
    rawset my_global_var .another_member 456
]])

In the python script, provide a class that handles the language via sublanguage_main:

(This is a complete example of what sublanguages can do, you can find a simpler example below

from pico_process import SubLanguageBase, is_identifier
from collections import Counter

class MySubLanguage(SubLanguageBase):
    # NOTE: all members are optional.

    # called to parse the sub-language from a string
    # (strings consist of raw pico-8 chars ('\0' to '\xff') - not real unicode)
    def __init__(self, str, on_error, **_):
        # our trivial language consists of space-separated tokens in newline-separated statements
        self.stmts = [stmt.split() for stmt in str.splitlines()]
        # we can report parsing errors:
        #on_error("Example")

    # these are utility functions for our own use:

    def is_global(self, token):
        # is the token a global in our language? e.g. sin / rectfill / g_my_global
        return is_identifier(token)

    def is_member(self, token):
        # is the token a member in our language? e.g. .my_member / .x
        return token.startswith(".") and self.is_global(token[1:])
        
    def is_assignment(self, stmt):
        return len(stmt) > 1 and stmt[1] == "<-" # our lang's assignment token

    # for --minify:

    # called to get all characters that won't get removed or renamed by the minifier
    # (aka, all characters other than whitespace and identifiers)
    # this is optional and doesn't affect correctness, but can slightly improve compressed size
    def get_unminified_chars(self, **_):
        for stmt in self.stmts:
            for token in stmt:
                if not self.is_global(token) and not self.is_member(token):
                    yield from token

    # called to get all uses of globals in the language's code
    def get_global_usages(self, **_):
        usages = Counter()
        for stmt in self.stmts:
            for token in stmt:
                if self.is_global(token):
                    usages[token] += 1
        return usages
        
    # called to get all uses of members (table keys) in the language's code
    def get_member_usages(self, **_):
        usages = Counter()
        for stmt in self.stmts:
            for token in stmt:
                if self.is_member(token):
                    usages[token[1:]] += 1
        return usages

    # for very advanced languages only, see test_input/sublang.py for details
    # def get_local_usages(self, **_):

    # called to rename all uses of globals/members/etc
    def rename(self, globals, members, **_):
        for stmt in self.stmts:
            for i, token in enumerate(stmt):
                if self.is_global(token) and token in globals:
                    stmt[i] = globals[token]
                elif self.is_member(token) and token[1:] in members:
                    stmt[i] = members[token[1:]]

    # called (after rename) to return a minified string
    def minify(self, **_):
        return "\n".join(" ".join(stmt) for stmt in self.stmts)

    # for --lint:

    # called to get globals defined within the sub-language's code
    # such globals can be used outside the sub-language too.
    def get_defined_globals(self, **_):
        for stmt in self.stmts:
            # our language only allows assignment to globals, so any assignment defines a global
            if self.is_assignment(stmt):
                yield stmt[0]

    # called to get globals used within the sub-language's code
    def get_used_globals(self, **_):
        for stmt in self.stmts:
            if self.is_assignment(stmt):
                stmt = stmt[2:] # don't return the assignment target, to get warnings if it isn't used

            for token in stmt:
                if self.is_global(token):
                    yield token

    # called to lint the sub-language's code
    def lint(self, builtins, globals, on_error, **_):
        for stmt in self.stmts:
            for token in stmt:
                if self.is_global(token) and token not in builtins and token not in globals:
                    on_error("Identifier '%s' not found" % token)
        # could do custom lints too

# this is called to get a sub-language class by name
def sublanguage_main(lang, **_):
    if lang == "evally":
        return MySubLanguage

Example - simple sub-language for table parsing

Often it's useful in pico-8 to define a simple sub-language to parse something like this:

"key1=val1,key2=val2,val3,val4"

To:

{key1="val1",key2="val2","val3","val4"}

Here, to minify properly, the keys (key1/key2) should be renamed as members, while the values should be left alone.

The custom python script:

from pico_process import SubLanguageBase
from collections import Counter

class SplitKeysSubLang(SubLanguageBase):
    # parses the string
    def __init__(self, str, **_):
        self.data = [item.split("=") for item in str.split(",")]

    # counts usage of keys
    # (returned keys are ignored if they're not identifiers)
    def get_member_usages(self, **_):
        return Counter(item[0] for item in self.data if len(item) > 1)

    # renames the keys
    def rename(self, members, **_):
        for item in self.data:
            if len(item) > 1:
                item[0] = members.get(item[0], item[0])

    # formats back to string
    def minify(self, **_):
        return ",".join("=".join(item) for item in self.data)

def sublanguage_main(lang, **_):
    if lang == "splitkeys":
        return SplitKeysSubLang

In the code:

-- (implementation of splitkeys omitted)
local table = splitkeys(--[[language::splitkeys]]"key1=val1,key2=val2,val3,val4")
?table.key1 -- "val1"
?table[1] -- "val3"

To run, use --script <path> as described before.

Advanced - access to the Syntax Tree

For really advanced usecases, you may want to have access to the Syntax Tree of your code (from a python script) in order to, e.g. do custom linting and analysis.

Keep in mind that the syntax tree and associated APIs are not fully documented here, and aren't guaranteed not to change in the future.

# this is called after your cart is parsed into a syntax tree, but before it is transformed for minification
def preprocess_syntax_main(cart, root, on_error, args, **_):
    from pico_parse import NodeType

    if args.lint: # do some custom linting, if linting was requested in the command line
        def pre_visit(node):
            # just as an example, add a lint error on any use of 'goto'
            if node.type == NodeType.goto:
                on_error("goto used", node)
            
            # the syntax tree format isn't really documented anywhere yet. you can:
            # - check examples of use in pico_lint.py
            # - print() nodes to see what they contain (ignores some attributes for better readability)
            # - search for the NodeType you're interested in, in pico_parse.py, to see what it contains

            # print(node)

        def post_visit(node):
            pass # just here as an example

        # visit the entire syntax tree, calling pre_visit before each node, and post_visit after each node
        # extra=True allows you to visit things not apparent in the source itself, such as:
        # implicit parameters, implicit _ENV when accessing globals, etc.
        root.traverse_nodes(pre=pre_visit, post=post_visit, extra=True)

To run, use --script <path> as described before.

You can check pico_lint.py for examples of how to use the syntax tree.

Picotron Support

The support is currently still experimental, and will remain as such at least while Picotron itself is experimental.

To use, use shrinkotron.py instead of shrinko8.py - the rest is largely the same.

Options specific for Shrinktron:

Cart manipulation features:

Notes: