Awesome

aws-open-data

Introduction

The AWS Open Data program hosts a lot of publicly available datasets. This repo compiles the list of all datasets on AWS as a CSV file and as a JSON file, making it easier to find and use them programmatically. The list is updated daily.

A complete list of AWS open datasets as individual YAML files is available here.

Usage

This repo provides the list of AWS open datasets in two formats:

• Tab separated values (TSV) file: aws_open_datasets.tsv
• JSON file: aws_open_datasets.json

The TSV file can be easily read into a Pandas DataFrame using the following code:

import pandas as pd

url = 'https://github.com/giswqs/aws-open-data/raw/master/aws_open_datasets.tsv'
df = pd.read_csv(url, sep='\t')
df.head()

Related Projects

• A list of open datasets on AWS: aws-open-data
• A list of open geospatial datasets on AWS: aws-open-data-geo
• A list of open geospatial datasets on AWS with a STAC endpoint: aws-open-data-stac
• A list of STAC endpoints from stacindex.org: stac-index-catalogs
• A list of geospatial datasets on Microsoft Planetary Computer: Planetary-Computer-Catalog
• A list of geospatial datasets on Google Earth Engine: Earth-Engine-Catalog
• A list of geospatial datasets on NASA's Common Metadata Repository (CMR): NASA-CMR-STAC
• A list of geospatial data catalogs: geospatial-data-catalogs