Awesome

sam4onnx

A very simple tool to rewrite parameters such as attributes and constants for OPs in ONNX models. Simple Attribute and Constant Modifier for ONNX.

https://github.com/PINTO0309/simple-onnx-processing-tools

Key concept

• Specify an arbitrary OP name and Constant type INPUT name or an arbitrary OP name and Attribute name, and pass the modified constants to rewrite the parameters of the relevant OP.
• Two types of input are accepted: .onnx file input and onnx.ModelProto format objects.
• To design the operation to be simple, only a single OP can be specified.
• Attributes and constants are forcibly rewritten, so the integrity of the entire graph is not checked in detail.
• Support for recursive search of subgraphs of If OP.

1. Setup

1-1. HostPC

### option
$ echo export PATH="~/.local/bin:$PATH" >> ~/.bashrc \
&& source ~/.bashrc

### run
$ pip install -U onnx \
&& python3 -m pip install -U onnx_graphsurgeon --index-url https://pypi.ngc.nvidia.com \
&& pip install -U sam4onnx

1-2. Docker

https://github.com/PINTO0309/simple-onnx-processing-tools#docker

2. CLI Usage

$ sam4onnx -h

usage:
    sam4onnx [-h]
    -if INPUT_ONNX_FILE_PATH
    -of OUTPUT_ONNX_FILE_PATH
    [-on OP_NAME]
    [-a NAME DTYPE VALUE]
    [-da DELETE_ATTRIBUTES [DELETE_ATTRIBUTES ...]]
    [-ic NAME DTYPE VALUE]
    [-os OUTPUT_NAME OUTPUT_SHAPE]
    [-n]

optional arguments:
  -h, --help
    show this help message and exit

  -if INPUT_ONNX_FILE_PATH, --input_onnx_file_path INPUT_ONNX_FILE_PATH
    Input onnx file path.

  -of OUTPUT_ONNX_FILE_PATH, --output_onnx_file_path OUTPUT_ONNX_FILE_PATH
    Output onnx file path.

  -on OP_NAME, --op_name OP_NAME
    OP name of the attributes to be changed.
    When --attributes is specified, --op_name must always be specified.
    e.g. --op_name aaa

  -a ATTRIBUTES ATTRIBUTES ATTRIBUTES, --attributes ATTRIBUTES ATTRIBUTES ATTRIBUTES
    Parameter to change the attribute of the OP specified in --op_name.
    If the OP specified in --op_name has no attributes,
    it is ignored. attributes can be specified multiple times.
    --attributes name dtype value dtype is one of
    "float32" or "float64" or "int32" or "int64" or "str".
    https://github.com/onnx/onnx/blob/main/docs/Operators.md

    e.g.
    --attributes alpha float32 [[1.0]]
    --attributes beta float32 [1.0]
    --attributes transA int64 0
    --attributes transB int64 0

  -da DELETE_ATTRIBUTES [DELETE_ATTRIBUTES ...], --delete_attributes DELETE_ATTRIBUTES [DELETE_ATTRIBUTES ...]
    Parameter to delete the attribute of the OP specified in --op_name.
    If the OP specified in --op_name has no attributes,
    it is ignored. delete_attributes can be specified multiple times.
    --delete_attributes name1 name2 name3
    https://github.com/onnx/onnx/blob/main/docs/Operators.md

    e.g. --delete_attributes alpha beta

  -ic INPUT_CONSTANTS INPUT_CONSTANTS INPUT_CONSTANTS, --input_constants INPUT_CONSTANTS INPUT_CONSTANTS INPUT_CONSTANTS
    Specifies the name of the constant to be changed.
    If you want to change only the constant,
    you do not need to specify --op_name and --attributes.
    input_constants can be specified multiple times.
    --input_constants constant_name numpy.dtype value

    e.g.
    --input_constants constant_name1 int64 0
    --input_constants constant_name2 float32 [[1.0,2.0,3.0],[4.0,5.0,6.0]]
    --input_constants constant_name3 float32 [\'-Infinity\']

  -os OUTPUT_SHAPES OUTPUT_SHAPES, --output_shapes OUTPUT_SHAPES OUTPUT_SHAPES
    Specifies the name of the output to be changed. output_shapes can be specified multiple times.
    --output_shapes output_name1 shape1
    --output_shapes output_name2 shape2

    e.g.
    --output_shapes output_name1 [1]
    --output_shapes output_name2 [1,3,224,224]

  -n, --non_verbose
    Do not show all information logs. Only error logs are displayed.

3. In-script Usage

$ python
>>> from sam4onnx import modify
>>> help(modify)

Help on function modify in module sam4onnx.onnx_attr_const_modify:

modify(
    input_onnx_file_path: Union[str, NoneType] = '',
    output_onnx_file_path: Union[str, NoneType] = '',
    onnx_graph: Union[onnx.onnx_ml_pb2.ModelProto, NoneType] = None,
    op_name: Union[str, NoneType] = '',
    attributes: Union[dict, NoneType] = None,
    delete_attributes: Union[List[str], NoneType] = None,
    input_constants: Union[dict, NoneType] = None,
    output_shapes: Optional[List] = None,
    non_verbose: Union[bool, NoneType] = False
) -> onnx.onnx_ml_pb2.ModelProto

    Parameters
    ----------
    input_onnx_file_path: Optional[str]
        Input onnx file path.
        Either input_onnx_file_path or onnx_graph must be specified.

    output_onnx_file_path: Optional[str]
        Output onnx file path.
        If output_onnx_file_path is not specified, no .onnx file is output.

    onnx_graph: Optional[onnx.ModelProto]
        onnx.ModelProto.
        Either input_onnx_file_path or onnx_graph must be specified.
        onnx_graph If specified, ignore input_onnx_file_path and process onnx_graph.

    op_name: Optional[str]
        OP name of the attributes to be changed.
        When --attributes is specified, --op_name must always be specified.
        Default: ''
        https://github.com/onnx/onnx/blob/main/docs/Operators.md

    attributes: Optional[dict]
        Specify output attributes for the OP to be generated.
        See below for the attributes that can be specified.

        {"attr_name1": numpy.ndarray, "attr_name2": numpy.ndarray, ...}

        e.g. attributes =
            {
                "alpha": np.asarray(1.0, dtype=np.float32),
                "beta": np.asarray(1.0, dtype=np.float32),
                "transA": np.asarray(0, dtype=np.int64),
                "transB": np.asarray(0, dtype=np.int64),
            }
        Default: None
        https://github.com/onnx/onnx/blob/main/docs/Operators.md

    delete_attributes: Optional[List[str]]
        Parameter to delete the attribute of the OP specified in --op_name.
        If the OP specified in --op_name has no attributes, it is ignored.
        delete_attributes can be specified multiple times.
        --delete_attributes name1 name2 name3
        https://github.com/onnx/onnx/blob/main/docs/Operators.md

        e.g.
        --delete_attributes alpha beta

    input_constants: Optional[dict]
        Specifies the name of the constant to be changed.
        If you want to change only the constant,
        you do not need to specify --op_name and --attributes.
        {"constant_name1": numpy.ndarray, "constant_name2": numpy.ndarray, ...}

        e.g.
        input_constants =
            {
                "constant_name1": np.asarray(0, dtype=np.int64),
                "constant_name2": np.asarray([[1.0,2.0,3.0],[4.0,5.0,6.0]], dtype=np.float32),
                "constant_name3": np.asarray([-np.inf], dtype=np.float32),
            }
        Default: None
        https://github.com/onnx/onnx/blob/main/docs/Operators.md

    output_shapes: Optional[List[int]]
        Specifies the name of the output_shapes to be changed.
        output_shapes can be specified multiple times.
        output_shapes = [
            ['output_name1', shape1],
            ['output_name2', shape2],
                    :
        ]
        e.g.
        output_shapes = [
            ['aaa', [1]],
            ['bbb', [1,3,224,224]],
        ]

    non_verbose: Optional[bool]
        Do not show all information logs. Only error logs are displayed.
        Default: False

    Returns
    -------
    modified_graph: onnx.ModelProto
        Mddified onnx ModelProto

4. CLI Execution

$ sam4onnx \
--input_onnx_file_path input.onnx \
--output_onnx_file_path output.onnx \
--op_name Transpose_17 \
--attributes perm int64 [0,1]

5. In-script Execution

from sam4onnx import modify

modified_graph = modify(
    onnx_graph=graph,
    op_name="Reshape_17",
    input_constants={"241": np.asarray([1], dtype=np.int64)},
    non_verbose=True,
)

6. Sample

6-1. Transpose - update perm and output_shapes

$ sam4onnx \
--input_onnx_file_path hitnet_sf_finalpass_720x1280_nonopt.onnx \
--output_onnx_file_path hitnet_sf_finalpass_720x1280_nonopt_mod.onnx \
--op_name Transpose_17 \
--attributes perm int64 [0,1] \
--output_shapes [128,256]

6-2. Mul - update Constant (170) - From: 2, To: 1

$ sam4onnx \
--input_onnx_file_path hitnet_sf_finalpass_720x1280_nonopt.onnx \
--output_onnx_file_path hitnet_sf_finalpass_720x1280_nonopt_mod.onnx \
--op_name Mul_5 \
--input_constants 170 float32 1

6-3. Reshape - update Constant (241) - From: [-1], To: [1]

$ sam4onnx \
--input_onnx_file_path hitnet_sf_finalpass_720x1280_nonopt.onnx \
--output_onnx_file_path hitnet_sf_finalpass_720x1280_nonopt_mod.onnx \
--op_name Reshape_34 \
--input_constants 241 int64 [1]

6-4. Support for recursive search of subgraphs of If OP.

Enhanced functionality to allow parameter rewriting even when the OP to be rewritten is in a hierarchy of multi-level subgraphs.

• main graph

  

• sub graph.1

  

• sub graph.2

  

7. Issues

https://github.com/PINTO0309/simple-onnx-processing-tools/issues