Awesome

Smithy Translate <!-- omit in toc -->

Tooling that enables converting to and from Smithy.

Note: this library is published to work on Java 8 and above. However, you will need to use Java 11 or above to work on the library as a contributor. This is due to some of the build flags that we use.

Table of Contents <!-- omit in toc -->

• formatter
  • CLI Usage
  • Capabilities and Design
• Alloy
• CLI
  • Installation
• OpenAPI
  • CLI Usage
  • Capabilities and Design
    • Primitives
    • Aggregate Shapes
      • Structure
      • Structures with Mixins
      • Untagged Union
      • Tagged Union
      • Discriminated Union
      • List
      • Set
      • Map
    • Constraints
      • Enum
      • Pattern
    • Service Shapes
      • Basic Service
      • Service with Error Responses
      • Operation with headers
      • Operation with multiple content types
    • Extensions
• JSON Schema
  • CLI Usage
  • Differences from OpenAPI
    • Default Values
    • Null Values
    • Maps
• Protobuf
  • CLI Usage
  • Capabilities and Design
  • Options
    • Stringly typed options
    • Example

formatter

CLI Usage

The smithytranslate CLI will recursively go through all child directories of the input directory provided and format any Smithy files it finds. The output

> smithytranslate format --help

Usage: smithytranslate format [--no-clobber] <path to Smithy file or directory containing Smithy files>...

validates and formats smithy files

Options and flags:
    --help
        Display this help text.
    --no-clobber
        dont overwrite existing file instead create a new file with the word 'formatted' appended so test.smithy -> test_formatted.smithy

Capabilities and Design

• The formatter is based off the ABNF defined at Smithy-Idl-ABNF
• The formatter assumes the file is a valid Smithy file and must be able to pass the Model Assembler validation , otherwise it will return an error
• use --no-clobber to create a new file to avoid overwriting the original file
• actual formatting rules are still WIP and will be updated as the formatter is developed

Alloy

Throughout smithytranslate you will see references to alloy. Alloy is a lightweight library that houses some common smithy shapes that are used across our open source projects such as smithy4s. This is to provide better interoperability between our tools at a lower cost to end users.

CLI

Installation

You will need to install coursier, an artifact fetching library, in order to install the CLI.

coursier install --channel https://disneystreaming.github.io/coursier.json smithytranslate

Run smithytranslate --help for usage information.

OpenAPI

CLI Usage

The smithytranslate CLI will recursively go through all child directories of the input directory provided and convert any openapi files ending with an extension of yaml, yml, or json.

> smithytranslate openapi-to-smithy --help

Usage: smithytranslate openapi-to-smithy --input <path> [--input <path>]... [--verboseNames] [--failOnValidationErrors] [--useEnumTraitSyntax] [--outputJson] <directory>

Take Open API specs as input and produce Smithy files as output.

Options and flags:
    --help
        Display this help text.
    --input <path>, -i <path>
        input source files
    --verbose-names
        If set, names of shapes not be simplified and will be as verbose as possible
    --validate-input
        If set, abort the conversion if any input specs contains a validation error
    --validate-output
        If set, abort the conversion if any produced smithy spec contains a validation error
    --enum-trait-syntax
        output enum types with the smithy v1 enum trait (deprecated) syntax
    --json-output
        changes output format to be json representations of the smithy models

Run smithytranslate openapi-to-smithy --help for more usage information.

Capabilities and Design

Because Smithy is a more constrained format than OpenAPI, this conversion is partial. This means that a best effort is made to translate all possible aspects of OpenAPI into Smithy and errors are outputted when something cannot be translated. When errors are encountered, the conversion still makes a best effort at converting everything else. This way, as much of the specification will be translated automatically and the user can decide how to translate the rest.

OpenAPI 2.x and 3.x are supported as input formats to this converter.

Below are examples of how Smithy Translate converts various OpenAPI constructs into Smithy.

Primitives

Aggregate Shapes

Structure

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    Testing:
      type: object
      properties:
        myString:
          type: string
        my_int:
          type: integer
      required:
        - myString

Smithy:

structure Testing {
 @required
 myString: String
 my_int: Integer
}

Required properties and nested structures are both supported.

Any properties in the input structure that begin with a number will be prefixed by the letter n. This is because smithy does not allow for member names to begin with a number. You can change this with post-processing if you want a different change to be made to names of this nature. Note that this extra n will not impact JSON encoding/decoding because we also attach the JsonName Smithy trait to these properties. The same thing happens if the member name contains a hyphen. In this case, hyphens are replaced with underscores and a jsonName trait is once again added. Note that if the field is a header or query parameter, the jsonName annotation is not added since httpHeader or httpQuery is used instead.

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    Testing:
      type: object
      properties:
        12_twelve:
          type: string
        X-something:
          type: string

Smithy:

structure Testing {
 @jsonName("12_twelve")
 n12_twelve: String
 @jsonName("X-something")
 X_something: String
}

Structures with Mixins

Smithy Translate will convert allOfs from OpenAPI into structures with mixins in smithy where possible. AllOfs in OpenAPI have references to other types which compose the current type. We refer to these as "parents" or "parent types" below. There are three possibilities when converting allOfs to smithy shapes:

1. The parent structures are only ever used as mixins

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    One:
      type: object
      properties:
        one:
          type: string
    Two:
      type: object
      properties:
        two:
          type: string
    Three:
      type: object
      allOf:
        - $ref: "#/components/schemas/One"
        - $ref: "#/components/schemas/Two"

Smithy:

@mixin
structure One {
 one: String
}

@mixin
structure Two {
  two: String
}

structure Three with [One, Two] {}

Here we can see that both parents, One and Two are converted into mixins and used as such on Three.

2. The parents structures are used as mixins and referenced as member targets

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    One:
      type: object
      properties:
        one:
          type: string
    Two:
      type: object
      allOf:
        - $ref: "#/components/schemas/One"
    Three:
      type: object
      properties:
        one:
          $ref: "#/components/schemas/One"

Smithy:

@mixin
structure OneMixin {
  one: String
}

structure One with [OneMixin] {}

structure Two with [OneMixin] {}

structure Three {
  one: One
}

Here One is used as a target of the Three$one member and is used as a mixin in the Two structure. Since smithy does not allow mixins to be used as targets, we have to create a separate mixin shape, OneMixin which is used as a mixin for One which is ultimately what we use for the target in Three.

3. One of the parents is a document rather than a structure

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    One:
      type: object
      properties: {}
    Two:
      type: object
      properties:
        two:
          type: string
    Three:
      type: object
      allOf:
        - $ref: "#/components/schemas/One"
        - $ref: "#/components/schemas/Two"

Smithy:

document One

structure Two {
  two: String
}

document Three

In this case, no mixins are created since none are ultimately used. Since One is translated to a document, Three must also be a document since it has One as a parent shape. As such, Two is never used as a mixin.

Untagged Union

The majority of oneOf schemas in OpenAPI represent untagged unions. As such, they will be tagged with the alloy#untagged trait. There are two exceptions to this: tagged unions and discriminated unions, shown in later examples.

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    Cat:
      type: object
      properties:
        name:
          type: string
    Dog:
      type: object
      properties:
        breed:
          type: string
    TestUnion:
      oneOf:
        - $ref: '#/components/schemas/Cat'
        - $ref: '#/components/schemas/Dog'

Smithy:

use alloy#untagged

structure Cat {
    name: String
}

structure Dog {
    breed: String
}

@untagged
union TestUnion {
    Cat: Cat,
    Dog: Dog
}

Tagged Union

Smithy Translate will convert a oneOf to a tagged union IF each of the branches of the oneOf targets a structure where each of those structures contains a single required property. Note that unions in smithy are tagged by default, so there is no trait annotation required here.

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    Number:
      type: object
      properties:
        num:
          type: integer
      required:
        - num
    Text:
      type: object
      properties:
        txt:
          type: string
      required:
        - txt
    TestUnion:
      oneOf:
        - $ref: '#/components/schemas/Number'
        - $ref: '#/components/schemas/Text'

Smithy:

structure Number {
    @required
    num: Integer,
}

structure Text {
    @required
    txt: String,
}

union TestUnion {
    num: Integer,
    txt: String
}

Although TestUnion is a tagged union that can be represented by directly targeting the Integer and String types, Text and Number are still rendered. This is because they are top-level schemas and could be used elsewhere.

Discriminated Union

A oneOf will be converted to a discriminated union IF it contains the discriminator field. Discriminated unions in Smithy will be denoted using the alloy#discriminated trait. The discriminated trait contains the name of the property that is used as the discriminator.

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    Cat:
      type: object
      properties:
        name:
          type: string
        pet_type:
          type: string
    Dog:
      type: object
      properties:
        breed:
          type: string
        pet_type:
          type: string
    TestUnion:
      oneOf:
        - $ref: '#/components/schemas/Cat'
        - $ref: '#/components/schemas/Dog'
      discriminator:
        propertyName: pet_type

Smithy:

use alloy#discriminated

structure Cat {
    name: String,
}

structure Dog {
    breed: String,
}

@discriminated("pet_type")
union TestUnion {
    Cat: Cat,
    Dog: Dog
}

List

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    StringArray:
      type: array
      items:
        type: string

Smithy:

list StringArray {
    member: String
}

Set

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    StringSet:
      type: array
      items:
        type: string
      uniqueItems: true

Smithy:

@uniqueItems
list StringSet {
    member: String
}

Map

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    StringStringMap:
      type: object
      additionalProperties:
        type: string

Smithy:

map StringStringMap {
    key: String,
    value: String
}

Constraints

Enum

Enums can be translated to either Smithy V1 or V2 syntax. You can control this using the useEnumTraitSyntax CLI flag.

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    Color:
      type: string
      enum:
        - red
        - green
        - blue

Smithy:

enum Color {
  red
  green
  blue
}

Or if using the useEnumTraitSyntax flag:

@enum([
 {value: "red"},
 {value: "green"},
 {value: "blue"}
])
string Color

Pattern

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths: {}
components:
  schemas:
    MyString:
      type: string
      pattern: '^\d{3}-\d{2}-\d{4}$'

Smithy:

@pattern("^\\d{3}-\\d{2}-\\d{4}$")
string MyString

Note that length, range, and sensitive traits are also supported, as indicated in the primitives table above.

Service Shapes

Basic Service

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths:
  /test:
    post:
      operationId: testOperationId
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ObjectIn'
      responses:
        '200':
          description: test
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ObjectOut'
components:
  schemas:
    ObjectIn:
      type: object
      properties:
        s:
          type: string
      required:
        - s
    ObjectOut:
      type: object
      properties:
        sNum:
          type: integer

If provided, such as above, the operationId will be used to inform the naming of the operation and the various shapes it contains.

Smithy:

use smithytranslate#contentType

service FooService {
    operations: [TestOperationId]
}

@http(method: "POST", uri: "/test", code: 200)
operation TestOperationId {
    input: TestOperationIdInput,
    output: TestOperationId200
}

structure ObjectIn {
    @required
    s: String
}

structure ObjectOut {
    sNum: Integer
}

structure TestOperationId200 {
    @httpPayload
    @required
    @contentType("application/json")
    body: ObjectOut
}

structure TestOperationIdInput {
    @httpPayload
    @required
    @contentType("application/json")
    body: ObjectIn
}

Service with Error Responses

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths:
  /test:
    get:
      operationId: testOperationId
      responses:
        '200':
          description: test
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Object'
        '404':
          description: test
          content:
            application/json:
              schema:
                type: object
                properties:
                  message:
                    type: string
components:
  schemas:
    Object:
      type: object
      properties:
        s:
          type: string
      required:
        - s

Smithy:

use smithytranslate#contentType

service FooService {
    operations: [TestOperationId]
}

@http(method: "GET", uri: "/test", code: 200)
operation TestOperationId {
    input: Unit,
    output: TestOperationId200,
    errors: [TestOperationId404]
}

structure Object {
    @required
    s: String
}

@error("client")
@httpError(404)
structure TestOperationId404 {
    @httpPayload
    @required
    @contentType("application/json")
    body: Body
}

structure Body {
    message: String
}

structure TestOperationId200 {
    @httpPayload
    @required
    @contentType("application/json")
    body: Object
}

Operation with headers

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths:
  /test:
    get:
      operationId: testOperationId
      parameters:
        - in: header
          name: X-username
          schema:
            type: string
      responses:
        '200':
          description: test
          headers:
            X-RateLimit-Limit:
              schema:
                type: integer
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Object'
components:
  schemas:
    Object:
      type: object
      properties:
        s:
          type: string
      required:
        - s

Smithy:

use smithytranslate#contentType

service FooService {
    operations: [TestOperationId]
}

@http(method: "GET", uri: "/test", code: 200)
operation TestOperationId {
    input: TestOperationIdInput,
    output: TestOperationId200
}

structure TestOperationIdInput {
    @httpHeader("X-username")
    X_username: String
}

structure Object {
    @required
    s: String,
}

structure TestOperationId200 {
    @httpPayload
    @required
    @contentType("application/json")
    body: Object,
    @httpHeader("X-RateLimit-Limit")
    X_RateLimit_Limit: Integer
}

Operation with multiple content types

Operations in OpenAPI may contain more than one content type. This is represented in smithy using a union with a special contentTypeDiscriminated trait. This trait indicates that the members of the union are discriminated from one another using the Content-Type header. Each member of the union is annotated with the contentType trait. This trait indicates which content type refers to each specific branch of the union.

OpenAPI:

openapi: '3.0.'
info:
  title: doc
  version: 1.0.0
paths:
  /test:
    post:
      operationId: testOperationId
      requestBody:
        required: true
        content:
          application/octet-stream:
            schema:
              type: string
              format: binary
          application/json:
            schema:
              type: object
              properties:
                s:
                  type: string
      responses:
        '200':
          description: test
          content:
            application/octet-stream:
              schema:
                type: string
                format: binary
            application/json:
              schema:
                type: object
                properties:
                  s:
                    type: string

Smithy:

use smithytranslate#contentTypeDiscriminated
use smithytranslate#contentType

service FooService {
    operations: [TestOperationId]
}

@http(method: "POST", uri: "/test", code: 200)
operation TestOperationId {
    input: TestOperationIdInput,
    output: TestOperationId200
}

structure TestOperationIdInput {
    @httpPayload
    @required
    body: TestOperationIdInputBody
}

structure TestOperationId200 {
    @httpPayload
    @required
    body: TestOperationId200Body
}

@contentTypeDiscriminated
union TestOperationId200Body {
    @contentType("application/octet-stream")
    applicationOctetStream: Blob,
    @contentType("application/json")
    applicationJson: TestOperationId200BodyApplicationJson
}

structure TestOperationId200BodyApplicationJson {
    s: String
}

@contentTypeDiscriminated
union TestOperationIdInputBody {
  @contentType("application/octet-stream")
  applicationOctetStream: Blob,
  @contentType("application/json")
  applicationJson: TestOperationIdInputBodyApplicationJson
}

structure TestOperationIdInputBodyApplicationJson {
  s: String
}

Extensions

OpenAPI extensions are preserved in the output Smithy model through the use of the openapiExtensions trait.

OpenAPI:

openapi: '3.0.'
info:
  title: test
  version: '1.0'
paths: {}
components:
  schemas:
    MyString:
      type: string
      x-float: 1.0
      x-string: foo
      x-int: 1
      x-array: [1, 2, 3]
      x-null: null
      x-obj:
        a: 1
        b: 2

Smithy:

use alloy.openapi#openapiExtensions

@openapiExtensions(
 "x-float": 1.0,
 "x-array": [1, 2, 3],
 "x-string": "foo",
 "x-int": 1,
 "x-null": null,
 "x-obj": {
   a: 1,
   b: 2
 }
)
string MyString

JSON Schema

CLI Usage

> smithytranslate json-schema-to-smithy --help

Usage: smithytranslate json-schema-to-smithy --input <path> [--input <path>]... [--verboseNames] [--failOnValidationErrors] [--useEnumTraitSyntax] [--outputJson] <directory>

Take Json Schema specs as input and produce Smithy files as output.

Options and flags:
    --help
        Display this help text.
    --input <path>, -i <path>
        input source files
    --verbose-names
        If set, names of shapes not be simplified and will be as verbose as possible
    --validate-input
        If set, abort the conversion if any input specs contains a validation error
    --validate-output
        If set, abort the conversion if any produced smithy spec contains a validation error
    --enum-trait-syntax
        output enum types with the smithy v1 enum trait (deprecated) syntax
    --json-output
        changes output format to be json representations of the smithy models

Run smithytranslate json-schema-to-smithy --help for all usage information.

Differences from OpenAPI

Most of the functionality of the OpenAPI => Smithy conversion is the same for the JSON Schema => Smithy one. As such, here we will outline any differences that exist. Everything else is the same.

Default Values

Default values from JSON Schema will be captured in the smithy.api#default trait.

JSON Schema:

{
 "$id": "test.json",
 "$schema": "http://json-schema.org/draft-07/schema#",
 "title": "Person",
 "type": "object",
 "properties": {
   "firstName": {
     "type": "string",
     "default": "Sally"
   }
 }
}

Smithy:

structure Person {
 @default("Sally")
 firstName: String
}

Null Values

JSON Schemas allows for declaring types such as ["string", "null"]. This type declaration on a required field means that the value cannot be omitted from the JSON payload entirely, but may be set to null. For example:

JSON Schema:

{
  "$id": "test.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Foo",
  "type": "object",
  "properties": {
    "bar": {
      "type": ["string", "null"]
    }
  },
  "required": ["bar"]
}

Smithy:

use alloy#nullable

structure Foo {
 @required
 @nullable
 bar: String
}

In most protocols, there is likely no difference between an optional field and a nullable optional field. Similarly, some protocols may not allow for required fields to be nullable. These considerations are left up to the protocol itself.

Maps

JSON Schema doesn't provide a first-class type for defining maps. As such, we translate a commonly-used convention into map types when encountered. When patternProperties is set to have a single entry, .*, we translate that to a smithy map type.

JSON Schema:

{
  "$id": "test.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "TestMap",
  "type": "object",
  "patternProperties": {
    ".*": {
      "type": "string"
    }
  }
}

Smithy:

map TestMap {
 key: String,
 value: String
}

Protobuf

CLI Usage

> smithytranslate smithy-to-proto --help

Usage: smithytranslate smithy-to-proto --input <path> [--input <path>]... [--dependency <string>]... [--repository <string>]... <directory>

Take Smithy definitions as input and produce Proto files as output.

Options and flags:
    --help
        Display this help text.
    --input <path>, -i <path>
        input source files
    --dependency <string>
        Dependencies that contains Smithy definitions.
    --repository <string>
        Specify repositories to fetch dependencies from.

Run smithytranslate smithy-to-proto --help for more usage information.

Capabilities and Design

The design of the smithy to protobuf translation follows the semantics defined in the alloy specification.

Options

Individual protobuf definitions file (.proto) can contain options. We support this feature using Smithy's metadata attribute.

There are a few importing things to notice

1. All options are defined under the metadata key proto_options
2. The value is an array. This is because Smithy will concatenate the arrays if the model contains multiple entries
3. Each entry of the array is an object where the keys are the namespace and the values are objects that represent the options
4. Entries for other namespaces are ignored (for example, demo in the example below)
5. The object that represents an option can only use String as value (see the example below). More detail below.

Stringly typed options

We used a String to represent the option such as "true" for a boolean and "\"demo\"" for a String because it's the simplest approach to cover all use cases supported by protoc. protoc supports simple types like you'd expect: bool, int, float and string. But it also supports identifier which are a reference to some value that protoc knows about. For example: option optimize_for = CODE_SIZE;. Using a String for the value allows us to model this, while keeping thing simple. It allows prevent the users from trying to use Arrays or Object as value for options.

Example

The following is an example:

Smithy:

$version: "2"

metadata "proto_options" = [{
  "foo": {
      "java_multiple_files": "true",
      "java_package": "\"foo.pkg\""
  },
  "demo": {
      "java_multiple_files": "true"
  }
}]

namespace foo

structure Foo {
  value: String
}

Proto:

syntax = "proto3";

option java_multiple_files = true;
option java_package = "foo.pkg";

package foo;

message Foo {
  string value = 1;
}