Home

Awesome

<p align="center"> <a href="https://github.com/pennsignals/aptos"><img src="https://rawgit.com/rightlag/d97e8d74d5bbb5b2e0cbe2938c40802d/raw/c957e9af39b1e6d94c1b5e823685084e7beba6ad/aptos.svg"></a> </p> <p align="center"> <a href="https://travis-ci.org/pennsignals/aptos"><img src="https://img.shields.io/travis/pennsignals/aptos.svg?style=flat-square" alt="Build Status"></a> <a href="https://coveralls.io/github/pennsignals/aptos"><img src="https://img.shields.io/coveralls/pennsignals/aptos.svg?style=flat-square" alt="Coverage Status"></a> <a href="https://pypi.org/project/aptos"><img src="https://img.shields.io/pypi/v/seven.svg?style=flat-square" alt="PyPI"></a> <a href="https://gitter.im/aptos-io/Lobby"><img src="https://img.shields.io/gitter/room/aptos/aptos-io.svg?style=flat-square" alt="Gitter"></a> </p>

Validate client-submitted data using JSON Schema documents and convert JSON Schema documents into different data-interchange formats.

Contents

Why aptos?

Installation

via pip

$ pip install aptos

via git

$ git clone https://github.com/pennsignals/aptos.git && cd aptos
$ python setup.py install

Usage

aptos supports the following capabilities:

usage: aptos [arguments] SCHEMA

aptos is a tool for validating client-submitted data using the JSON Schema
vocabulary and converts JSON Schema documents into different data-interchange
formats.

positional arguments:
  schema              JSON document containing the description

optional arguments:
  -h, --help          show this help message and exit

Arguments:
  {validate,convert}
    validate          Validate a JSON instance
    convert           Convert a JSON Schema into a different data-interchange
                      format

More information on JSON Schema: http://json-schema.org/

Data Validation

Here is a basic example of a JSON Schema:

{
    "title": "Person",
    "type": "object",
    "properties": {
        "firstName": {
            "type": "string"
        },
        "lastName": {
            "type": "string"
        },
        "age": {
            "description": "Age in years",
            "type": "integer",
            "minimum": 0
        }
    },
    "required": ["firstName", "lastName"]
}

Given a JSON Schema, aptos can validate client-submitted data to ensure that it satisfies a certain number of criteria.

JSON Schema Validation keywords such as minimum and required can be used to impose requirements for successful validation of an instance. In the JSON Schema above, both the firstName and lastName properties are required, and the age property MUST have a value greater than or equal to 0.

Valid Instance :heavy_check_mark:Invalid Instance :heavy_multiplication_x:
{"firstName": "John", "lastName": "Doe", "age": 42}{"firstName": "John", "age": -15} (missing required property lastName and age is not greater than or equal to 0)

aptos can validate client-submitted data using either the CLI or the API:

Data Validation CLI

$ aptos validate -instance INSTANCE SCHEMA

Arguments:

Example - macOS:

$ aptos validate -instance '{"firstName": "John"}' person.json

Example - Windows:

> aptos validate -instance "{\"firstName\": \"John\"}" person.json
Successful Validation :heavy_check_mark:Unsuccessful Validation :heavy_multiplication_x:

Data Validation API

import json

from aptos.parser import SchemaParser
from aptos.visitor import ValidationVisitor


with open('/path/to/schema') as fp:
    schema = json.load(fp)
component = SchemaParser.parse(schema)
# Invalid client-submitted data (instance)
instance = {
  'firstName': 'John'
}
try:
    component.accept(ValidationVisitor(instance))
except AssertionError as e:
    print(e)  # instance {'firstName': 'John'} is missing required property 'lastName'

Structured Message Generation

Given a JSON Schema, aptos can generate different structured messages.

:warning: Note: The JSON Schema being converted MUST be a valid JSON Object.

Supported Data-Interchange Formats

FormatSupportedNotes
Apache Avro:heavy_check_mark:
Protocol Buffers:heavy_multiplication_x:Planned for future releases
Apache Thrift:heavy_multiplication_x:Planned for future releases
Apache Parquet:heavy_multiplication_x:Planned for future releases

Avro

Using the Person schema in the previous example, aptos can convert the schema into the Avro data-interchange format using either the CLI or the API.

aptos maps the following JSON schema types to Avro types:

JSON Schema TypeAvro Type
stringstring
booleanboolean
nullnull
integerlong
numberdouble
objectrecord
arrayarray

JSON Schema documents containing the enum validation keyword are mapped to Avro enum symbols attribute.

JSON Schema documents with the type keyword as an array are mapped to Avro Union types.

Data-Interchange CLI

$ aptos convert -format FORMAT SCHEMA

Arguments:

<p align="center"> <img src="https://user-images.githubusercontent.com/2184329/29071365-3d6e7504-7c11-11e7-959e-abcfe15f5e96.png" width="600"> </p>

Data-Interchange API

import json

from aptos.parser import SchemaParser
from aptos.schema.visitor import AvroSchemaVisitor


with open('/path/to/schema') as fp:
    schema = json.load(fp)
component = SchemaParser.parse(schema)
record = component.accept(AvroSchemaVisitor())
print(json.dumps(record, indent=2))

The above code generates the following Avro schema:

{
  "type": "record",
  "fields": [
    {
      "doc": "",
      "type": "string",
      "name": "lastName"
    },
    {
      "doc": "",
      "type": "string",
      "name": "firstName"
    },
    {
      "doc": "Age in years",
      "type": "long",
      "name": "age"
    }
  ],
  "name": "Person"
}

Testing

All unit tests exist in the tests directory.

To run tests, execute the following command:

$ python setup.py test

Additional Resources

Future Considerations

Maintainers

Jason Walsh
Jason Walsh

Contributing

Contributions welcome! Please read the contributing.json file first.

Join our Slack channel!

License

Apache 2.0 © Penn Signals