Awesome

Processing Extension Specification

• Title: Processing
• Identifier: https://stac-extensions.github.io/processing/v1.2.0/schema.json
• Field Name Prefix: processing
• Scope: Item, Collection
• Extension Maturity Classification: Candidate
• Owner: @emmanuelmathot

Processing metadata is considered to be data that indicate from which processing chain a data originates and how the data itself has been produced. Overall, it helps to increase traceability and search among processing levels and multiple algorithm versions.

Often, data items are the result of one or more waterfall processing pipeline. Tracing information such as the processing facility, the algorithm version or the processing date helps in the data version management.

This extension applies to STAC Items and STAC Collections. As these processing information are often closely bound to the Collection level and therefore are shared across all items, it is recommended adding the fields to the corresponding STAC Collection.

• Examples:
  • Item example: Shows the basic usage of the extension in a STAC Item
  • Collection example: Shows the basic usage of the extension in a STAC Collection
• JSON Schema
• Changelog

Fields

The fields in the table above can be used in these parts of STAC documents:

• Catalogs
• Collections
• Collection Provider
• Item Properties (incl. Summaries in Collections)
• Assets (for both Collections and Items, incl. Item Asset Definitions in Collections)
• Links

In more detail, the following restrictions apply:

1. Items:

   • The fields are usually placed in the properties. At least one field is required to be present.
   • Additionally, STAC allows all fields to be used in the Asset Object.

2. Collections:

   • The fields are usually placed in the Provider Objects for the providers that have the role producer or processor assigned. They don't need to be provided for all providers of the respective role.
   • The fields can also be used in summaries, Collection assets or Item asset definitions (item_assets). Please note that the JSON Schema is not be able to validate the values of Collection summaries.

If the extension is given in the stac_extensions list, at least one of the fields must be specified in any of the given places listed above.

Processing Date Time

The time of the processing can be specified as a global field in processing:datetime, but it can also be specified directly and individually via the created properties of the target asset as specified in the STAC Common metadata.

created in Item properties describes the STAC metadata creation and in Assets it describes the creation of the data files. Thus the timestamps provided in Item Properties for created and processing:datetime may differ. As Item properties are easier to be indexed and used for filtering purposes, processing:datetime exists. created and processing:datetime should usually be the same value in Assets and as such processing:datetime can usually be omitted.

Version Numbers

Three fields exist for version numbers:

• processing:software
• processing:version
• version (in the Version extension)

The different fields exist to give data providers more flexibility depending on their needs.

In Item Properties:

• processing:version is useful if a single version number is available for the metadata or data that users should be able to filter on. A popular example for this is the processing baseline in Sentinel missions.
• processing:software is used if the software libraries/tools are important to know, but it's not important to filter on them. They are mostly informative and important to be complete for reporducibility purposes. Thus, the values in the object can not just be version numbers, but also be e.g. tag names, commit hashes or similar. For example, you could expose a simplified version of the Pipfile.lock (Python) or package-lock.json (NodeJS). If you need more information, you could also link to such files via the relation type processing-software.
• version is usually not used in the context of processing and describes the version of the metadata.

Linking the Items

In Items that declare this processing extension, it is recommended to add one or more Links with derived_from or via relationships to the eventual source metadata & data used in the processing. They could be used to trace back the processing history of the dataset.

Suggested Processing Levels

The processing:level is the name that is commonly used to refer to that processing level properties. The table below shows some processing level used by the industry for some data product.

Each level represents a step in the abstraction process by which data relevant to physical information (raw, level 0, level 1) are turned into data relevant to geo physical information (level 2, level 3), and finally turned into data relevant to thematic information (level 4)

This list is not exhaustive and can be extended with the processing level specific to a data product.

Expression Object

Potential expression formats with examples:

Relation types

The following types should be used as applicable rel types in the Link Object.

Contributing

All contributions are subject to the STAC Specification Code of Conduct. For contributions, please follow the STAC specification contributing guide Instructions for running tests are copied here for convenience.

Running tests

The same checks that run as checks on PR's are part of the repository and can be run locally to verify that changes are valid. To run tests locally, you'll need npm, which is a standard part of any node.js installation.

First you'll need to install everything with npm once. Just navigate to the root of this repository and on your command line run:

npm install

Then to check markdown formatting and test the examples against the JSON schema, you can run:

npm test

This will spit out the same texts that you see online, and you can then go and fix your markdown or examples.

If the tests reveal formatting problems with the examples, you can fix them with:

npm run format-examples