Home

Awesome

niceshade

A wrapper for the Microsoft DirectXShaderCompiler and SPIRV-Cross

Build status

See the reference documentation for libniceshade at https://wiki.gpfault.net/docs/niceshade/

User Manual

Table of Contents

<a name="intro"></a>

Introduction

niceshade is a library and a command-line tool that transforms HLSL code into shaders for various graphics APIs. Presently, the following APIs can be targeted:

The input HLSL files may contain definitions of several entry points for different shader stages. The entry points can be configured into a single rendering pipeline (with additional options, if desired) using a special directive. For each of these configurations (called techniques), the tool will generate platform-specific shaders.

As an example, here is a shader that calculates the relative luminance of each pixel in an image. For demonstration purposes, it allows to optionally apply gamma correction to input and/or output.

// The comments below are recognized by niceshade as technique definitions.
//T: relative-luminance vs:VSMain ps:PSMain define:OUTPUT_NEEDS_GAMMA_CORRECTION=1 define:INPUT_NEEDS_GAMMA_CORRECTION=1
//T: relative-luminance-srgb-texture vs:VSMain ps:PSMain define:OUTPUT_NEEDS_GAMMA_CORRECTION=1
//T: relative-luminance-srgb-framebuffer vs:VSMain ps:PSMain define:INPUT_NEEDS_GAMMA_CORRECTION=1
//T: relative-luminance-srgb-texture-and-framebuffer vs:VSMain ps:PSMain

float4 VSMain(uint vid : SV_VertexID) : SV_POSITION { // vertex shader
  const float2 fullscreen_triangle_verts[] = {
    float2(-1.0, -1.0), float2(3.0, -1.0), float2(-1.0,  3.0)
  };
  return  float4(fullscreen_triangle_verts[vid % 3], 0.0, 1.0);
}

uniform Texture2D img;

float4 PSMain(float4 frag_coord : SV_POSITION) : SV_TARGET { // pixel shader
  const float gamma = 2.2;
  uint img_width, img_height;
  img.GetDimensions(img_width, img_height);
  float3 color = img.Load(int3(int2(frag_coord.xy) % int2(img_width, img_height), 0)).rgb;
#if defined(INPUT_NEEDS_GAMMA_CORRECTION)
  color = pow(color, gamma);
#endif
  float relative_luminance = dot(float3(0.2126, 0.7152, 0.0722), color);
#if defined(OUTPUT_NEEDS_GAMMA_CORRECTION)
  relative_luminance = pow(relative_luminance, 1.0 / gamma);
#endif
  return float4(float3(relative_luminance, relative_luminance, relative_luminance), 1.0);
}

In addition to generating shaders, the tool captures and writes out the information about resources (textures, buffers, etc.) used by each technique defined in the input file. This information can be used by the application for various purposes, such as streamlining Vulkan pipeline layout creation.

This tool is powered by Microsoft DirectXShaderCompiler and SPIRV-Cross.

<a name="building"></a>

Obtaining the Source Code and Building

You will need to have git and cmake installed on your system. On Windows, building with compilers other than MSVC may not work.

Execute the following command to clone the project's repository:

git clone https://github.com/nicebyte/niceshade.git

Once the cloning process is complete, execute the following commands from the root of the repository:

mkdir build
cd build
cmake .. -Ax64

This will generate project files specific to your system in the build folder. After building the generated project, the niceshade binary can be found in the repository's root folder.

<a name="running"></a>

Running

To transform an input HLSL file to platform-specific shaders, execute:

niceshade <input file name> <options>

Valid command line options are:

Shaders will be generated for each of the techniques specified in the input file and each of the targets specified in the command line options.

For example, the following line will produce OpenGL 4.3 and Metal 1.2 shaders for each technique defined in input.hlsl, in the generated_shaders/ subfolder:

niceshade input.hlsl -O generated_shaders/ -t gl430 -t msl12

<a name="techniques"></a>

Defining Techniques

Techniques are defined using a special comment:

//T: <technique name> <tags>

The technique name may include alphanumeric characters, underscores (_) and dashes (-).

A tag is a name-value pair separated by a colon. For example, vs:VSMain is a tag, vs is the tag name and VSMain is the value.

The following tag names are valid:

<a name="header-file"></a>

Generated Header File

Using the -h command line option, you may specify a special C++ header file to be generated as part of the shader compilation process. The said file shall contain named constants for all descriptor bindings and sets used by different techniques defined in the input. The constants for each technique are put into their own namespace, named after the technique (hyphens in tenchnique names are replaced by underscores to get valid C++ identifiers). Additionally, you may put the entire contents of the generated header into another namespace, specified by the -n command line option.

Below is an example of an input file and the generated header it produces.

Input file:

//T: imgui ps:PSMain vs:VSMain

struct ImGuiVSInput {
  float2 position : ATTR0;
  float2 uv : TEXCOORD0;
  float4 color : COLOR0;
};

struct ImGuiPSInput {
  float4 position : SV_POSITION;
  float2 uv : TEXCOORD0;
  float4 color : COLOR0;
};

cbuffer MatUniformBuffer : register(b0){
  float4x4 u_Projection;
}

[[vk::binding(1, 0)]] uniform Texture2D u_Texture;
[[vk::binding(2, 0)]] uniform sampler u_Sampler;

ImGuiPSInput VSMain(ImGuiVSInput input) {
  ImGuiPSInput output = {
    mul(u_Projection, float4(input.position, 0.0, 1.0)),
    input.uv,
    input.color
  };
  return output;
}

float4 PSMain(ImGuiPSInput input) : SV_TARGET {
  return input.color * u_Texture.Sample(u_Sampler, input.uv);
}

Generated header:

/*auto-generated, do not edit*/
#pragma once
namespace shader_consts {
namespace imgui {
  static constexpr int u_Sampler_Binding = 2;
  static constexpr int u_Sampler_Set = 0;
  static constexpr int u_Texture_Binding = 1;
  static constexpr int u_Texture_Set = 0;
  static constexpr int MatUniformBuffer_Binding = 0;
  static constexpr int MatUniformBuffer_Set = 0;
}
}

<a name="pipeline-metadata"></a>

Pipeline Metadata

For each technique defined in the input file, niceshade will produce a corresponding .pipeline file, which contains the following information:

.pipeline files are binary. Code for parsing the binary format is provided in the metadata_parser subfolder of the source code repository. Alternatively, .pipeline files can be converted to human-readable JSON using the display_metadata utility, the source code for which is provided in the samples subfolder of the repository. A detailed description of the metadata file format is provided below.

<a name="vk-hlsl"></a>

Using Vulkan features from HLSL

You may choose to use the Vulkan binding model explicitly and assign descriptor sets and bindings like this:

[[vk::binding(1, 0)]] uniform Texture2D tex; // assign tex to set 0 binding 1
[[vk::binding(2, 0)]] uniform sampler samp;  // assign tex to set 0 binding 2

You may use specialization constants as well:

[[vk::constant_id(1)]] const float specConstFloat = 1.5;

See here for more details.

<a name="metadata-format"></a>

Pipeline Metadata File Format

For each technique described in the input file, niceshade emits a file containing information that can be leveraged to simplify pipeline creation. This data includes:

A detailed description of the file's format follows.

General Conventions

A pipeline metadata file (referred to simply as "file" henceforth) is broken into records. A record is a sequence of fields and raw byte blocks. A field is a 4-byte unsigned integer in network byte order. A raw byte block consists of a header and a body. The body of a raw byte block is an arbitrary sequence of bytes. The length of the body is always a multiple of 4 bytes. The header of a raw byte block is two 32-bit unsigned integers in network byte order. The first is always equal to 0xffffffff and signifies the beginning of a raw byte block. The second specifies the length of the block's body divided by 4.

A record's type is defined by its layout. The following types of records are defined:

A detailed description of each record type follows.

The HEADER Record Type

The very first record in a file is always of the HEADER type. There is always only one instance of a HEADER record in a file.

A HEADER record contains the following fields, in this exact order:

The ENTRYPOINTS Record Type

This record type provides the names of entry points for individual shader stages involved in the technique.

The first field in this record, num_entrypoints, contains the number of entrypoints, one per shader stage. It is followed by a sequence of num_entrypoints entrypoint descriptions.

An entrypoint description consists of a field, type which indicates the type of the shader stage the entrypoint is for (0 for vertex shader, 1 for fragment shader), and a raw byte block containing a null-terminated entrypoint name.

The PIPELINE_LAYOUT Record Type

This record type provides information about descriptor sets, descriptor types and bindings used by the pipeline. There is always only one instance of a PIPELINE_LAYOUT record in a file.

A PIPELINE_LAYOUT record contains the following fields, in this exact order:

The SEPARATE_TO_COMBINED_MAP Record Type

Some platforms do not have full separation between textures and samplers. For example, in OpenGL, in order to sample a texture with a particular sampler, both the texture and the sampler need to be bound to the same texture unit by the CPU-side code. This is in contrast to HLSL, which allows specifying the sampler to use directly from the shader code.

To address this discrepancy, each unique texture-sampler pair used by the source HLSL generates a "synthetic" combined texture/sampler in the output. Each separate texture and sampler is then mapped to a set of auto-generated combined texture/samplers that it is used in.

A SEPARATE_TO_COMBINED_MAP record contains the following fields, in this exact order:

The USER_METADATA Record Type

This record stores any additional user metadata specified by meta: tags in the technique description.

The USER_METADATA record has only one field, num_metas. Following the field are num_metas pairs of raw byte blocks. The first block in a pair stores the user-provided key, and the second stores the user-provided value (both are null-terminated strings).

The THREADGROUP_SIZE Record Type

This record has meaning only for compute shaders. It stores the threadgroup size declared by the shader. For other shader types, this record is present, but contains zeros. The record has 3 fields, each corresponding to the threadgroup size in X, Y and Z dimensions accordingly.