Awesome

Atmoshack2018

Join EUMETSAT, ECMWF, FMI and University of Helsinki scientists for a weekend hackathon working with atmospheric data at the Finnish Meteorological Service in Helsinki on 16-18 November 2018.

This Github repository contains the resources for Copernicus Hackathon 2018 in Helsinki, it can be used to share code. More info at

https://ultrahack.org/atmoshack2018

https://www.eumetsat.int/website/home/News/ConferencesandEvents/DAT_4070861.html?lang=EN

Table of contents

1. Challenge
2. Dataset offering
3. Platform offering
4. Solution Domains
5. Background
6. Data Access
7. Useful Links

Challenge: Hack the Atmosphere! <a name="challenge"></a>

Urban air pollution poses a significant threat to human health and the quality of life of millions of people worldwide. Nine out of ten people are estimated to breathe air containing high levels of pollutants, causing around 7 million deaths every year. More than 90% of air pollution-related deaths occur in the developing world. Many of these areas have no surface air-quality monitoring networks and rely mainly on modelling and satellite information. Besides air pollution, there are other aspects of atmospheric composition that can affect health: UV radiation, which strongly depends on the stratospheric ozone layer, is another example. Urban air pollution poses a significant threat to human health and the quality of life of millions of people worldwide. Nine out of ten people are estimated to breathe air containing high levels of pollutants, causing around 7 million deaths every year. More than 90% of air pollution-related deaths occur in the developing world. Many of these areas have no surface air-quality monitoring networks and rely mainly on modelling and satellite information.

Your challenge is to create solutions that help people in their daily lives to reduce their exposure to pollutants and UV radiation. To solve this challenge, you are encouraged to use Copernicus atmospheric monitoring data.

Various approaches are welcome: Create new solutions or add atmosphere-smart features to existing solutions. Propose innovative data visualisations for academic purposes or for raising awareness around the atmospheric environment. Mobile, web apps, platforms or hardware - it’s your call how to improve the quality of life for many!

Dataset offering <a name="datasets"></a>

An overview of the complete set of datasets available for this event is listed below

Platform offering <a name="platform"></a>

WEkEO platform wekeo.eu provides access to Copernicus datasets and cloud based processing resources. Get to know the capabilities of WEkEO from here

You will be offered VM access by participating on the AtmosHack2018 here

Solution domains <a name="solution"></a>

• Air-quality apps
• Traffic and navigation
• Geological: volcanic eruptions, dust and smoke in the atmosphere
• Cleantech for industries, factories and agriculture
• Smart city solutions

Background <a name="bg"></a>

Copernicus AtmosHack is funded by the EU’s Copernicus Programme and has been organised through a partnership of EUMETSAT, the Copernicus Atmosphere Monitoring Service (CAMS), the Finnish Meteorological Service (FMI), and the University of Helsinki.

Data Access <a name="data"></a>

Object Storage End Point http://atmoshack.obs.eu-de.otc.t-systems.com/

list all product, for example, of CAMS Air Pollution

curl http://atmoshack.obs.eu-de.otc.t-systems.com/?prefix=06-CAMS-AirPollution

download a file

curl -O atmoshack.obs.eu-de.otc.t-systems.com/01-GOME_PMAP/M01-GOME/2017/01/M01-GOME-GOMPMA02-NA-2.0-201701003859.000000000Z-20170101022528-1293482-1.nc

Alternatively the data can be accessed with commonly used Object Storage tools, using the access keys below (AK/SK):

S3cmd https://s3tools.org/s3cmd
S3FS https://linux.die.net/man/1/s3fs

CAMS Data Access

Access CAMS data via the ECMWF MARS archive - Web-API

Retrieve ECMWF key

• Self-register at http://apps.ecmwf.int/registration

• Login at https://apps.ecmwf.int/auth/login

• Retrieve your API key at https://api.ecmwf.int/v1/key/

• Specify the ECMWFDataServer with your url, key and email information

Install the ecmwfapi python library

pip install https://software.ecmwf.int/wiki/download/attachments/56664858/ecmwf-api-client-python.tgz

If you cannot run the pip commands, just download the ecmwf-api-client-python.tgz library. Extract its content and copy the module ecmwfapi to a directory pointed by the environment variable PYTHONPATH

Execute a MARS request and download data either as GRIB or netCDF

NOTE: per default, ECMWF data are on a gaussian grid with longitudes going from 0 to 360 degrees. It can be reprojected to a regular geographic latitude-longitude grid. If a reprojection is wished, the key 'grid' with the respective latitude and longitude resolution has to be specified. The same applies for specifying a longitude range of -180 to 180. The key area can be set.

#!/usr/bin/env python 

from ecmwfapi import ECMWFDataServer

server = ECMWFDataServer(url="https://api.ecmwf.int/v1", key="...", email="...@...")

server.retrieve({
       'stream': "oper",
       'levtype': "sfc",
       'param': "167",
       'dataset': "interim",
       'step': "0",
       'grid': "0.5/0.5",
       'area': "90/-180/-90/179.5",
       'time': "00/06/12/18",
       'date': "2014-07-01/to/2014-07-31",
       'type': "an",
       'class': "ei",
       'format': "netcdf",
       'target': "test.nc"
        })

Useful Links <a name="useful"></a>

How to Visualize NetCDF data - YouTube Video

CAMS: https://atmosphere.copernicus.eu/

Example notebook: https://gist.github.com/erget/467dba7082d31d73b20f3b5e90e740af

GDAL: http://www.gdal.org/

Jupyter: https://jupyter.org/

Scipy stack, including matplotlib and numpy: https://www.scipy.org/

Info on netCDF tool: https://www.unidata.ucar.edu/software/netcdf/

QGIS: https://www.qgis.org/en/site/

Xarray: http://xarray.pydata.org/en/stable/

Scitools: https://scitools.org.uk/