Awesome

HMS-Smoke

NOAA's Hazard Mapping System (HMS)

The HMS Smoke Explorer allows end-users to visualize NOAA's Hazard Mapping System (HMS) smoke and fire products, MODIS aerosol optical depth, and GOES-East/West RGB imagery. Since 2005, NOAA analysts have been inspecting satellite imagery (e.g. GOES, MODIS, VIIRS) and manually outlining the extent of smoke across North America, classified into three density categories: light, medium, and heavy, to produce the HMS smoke product. A corresponding HMS fire product includes active fire detections from multiple satellites/sensors (e.g., MODIS, VIIRS, GOES, AVHRR) with quality control by the analysts.

The latest date available in the HMS Smoke Explorer is December 13, 2024.

Tool Capabilities

• Make an animation looping GOES-East/West RGB imagery. Right-click on the GIF and select "Save Image As..." to save the animation to your local storage.
• View satellite-derived (MODIS) and model forecast (CAMS) aerosol optical depth and surface PM_2.5 with HMS smoke.
• View HMS smoke text descriptions and summary statistics.

Datasets on HMS Smoke Explorer

The HMS Smoke Explorer covers the time range of the HMS Smoke Product (2005/08-present). Ancillary datasets for visualization are listed below.

HMS on Google Earth Engine

The processed HMS smoke product can be used for analysis on Google Earth Engine (EE) and downloaded as yearly files.

// read HMS smoke polygons and active fires by year
var inYear = 2020;
var HMS = ee.FeatureCollection('projects/GlobalFires/HMS/Smoke_Polygons/HMS_' + inYear);
var HMS_Fire = ee.FeatureCollection('projects/GlobalFires/HMS/Fire_Points/HMS_Fire_' + inYear);

// filter by month and day
var inMonth = 8;
var inDay = 1;
var HMS_day = HMS.filter(ee.Filter.eq('Month',8)).filter(ee.Filter.eq('Day',1));
var HMSFire_day = HMS_Fire.filter(ee.Filter.eq('Month',8)).filter(ee.Filter.eq('Day',1));

Map.addLayer(HMS_day);
Map.addLayer(HMSFire_day, {color:'red'});
Map.centerObject(HMS_day);

print(HMS_day);
print(HMSFire_day);

HMS Smoke Product

* only relevant for 2005-2010, so HMS files for only those years have these columns on EE

Notes:

• The start and end time represent the time bounds of the satellite images used to draw the HMS polygons, not the actual persistence duration of the smoke plume
• In the tool, duration is used to estimate the annual contribution of each smoke density category at a particular location
• For QA = 1, out-of-bounds coordinates with y > 90 or y < -90 are removed (removes some anomalous coordinates), x < -180 is forced to be x = -180 as this may lead to disfigured polygons on EE
• For QA = 2, the first coordinate is repeated at the end of the list of coordinates to close the polygon ring

Caveats

• The HMS smoke product represents smoke as seen from satellites. In some places, smoke may be aloft and may not affect surface air quality. This is particularly true for light smoke.
• Use caution when assessing trends in smoke using the HMS smoke product outside of CONUS. Note the lower spatial coverage in earlier years, which can be seen from the artificial boundaries in annual aggregates.
• The HMS 'Duration' is calculated from the start and end times of satellite images used to outline the smoke. Thus, it is not an estimate of the true smoke duration. HMS analysts outline smoke using only daytime satellite imagery and generally analyze heavy smoke twice per day in the morning and late afternoon.
• HMS smoke polygons in 2005-2007, 2009, and some in 2008 and 2010 are not classified into smoke density classes (light, medium, heavy). We used random forest modeling to assign densities to all such polygons.
• HMS smoke polygons with bad geometries and throws an error in R (i.e. drawn as lines rather than polygons, edges crossing edges) have been removed.
• GOES-16/East became operational on December 18, 2017, GOES-17/West on February 12, 2019, and GOES-18/West on January 4, 2023 (replacing GOES-17/West). Note these dates when selecting the GOES RGB images.

Summary Stats and Quality Control

Number of HMS polygons in each year, and how many are invalid after processing in R. The number of smoke polygons with gapfilled densities are also shown below.

Missing Dates

20050809,20050810,20060327,20060401,20060714,20060715,20061104,20070331,20070821,20080123,20080124,20080305,20081005,20090130,20090408,20121007,20150602,20150820,20160306,20161112,20170427,20170531,20170601,20170622,20170718,20190710,20190810

Caveats

• The availability of the satellites is not uniform throughout the record. For example, recent years have higher-resolution active fire detections from VIIRS, which is more capable at detecting small fires.
• The geolocation error for active fire detections varies among different satellites/sensors (e.g., 10s-100s m for VIIRS and up to 1 km for GOES) (see NOAA's HMS FAQ)
• There are missing HMS fire data (no .zip files) on some days.
• Some entries have malformed and/or missing date/time.

HMS Fire Product

Summary Stats and Quality Control

Number of HMS active fires from various satellites and missing days since April 1, 2003. MODIS includes Aqua and Terra, VIIRS includes S-NPP, NOAA-20, and NOAA-21, GOES includes GOES-East and GOES-West, and AVHRR includes NOAA-15 to NOAA-19 and METOP-A/B/2.

Missing Dates

20030501,20030502,20030503,20030504,20030505,20030506,20030507,20030508,20030509,20030510,20030511,20030512,20030513,20030514,20030515,20030516,20030517,20030518,20030519,20030520,20030521,20030522,20030523,20030524,20030525,20030526,20030527,20030528,20030529,20030530,20030531,20030601,20030602,20030603,20030604,20030605,20030606,20030607,20030608,20030609,20030610,20030611,20030612,20030613,20030614,20030615,20030711,20030731,20030830,20030831,20030905,20030907,20030914,20031102,20031210,20031215,20040225,20040402,20040624,20041209,20050511,20050624,20050626,20050703,20050706,20060327,20060401,20060714,20060715,20061104,20070331,20070821,20090130,20090408,20140705,20150602,20150820,20160306,20161108,20161112,20161209,20170427,20170531,20170601,20170622,20170718,20180501,20181220,20181230,20190225,20190709,20190710

Basic Code for Processing HMS Products

<b>Folder Structure</b>:

HMS/ 
	Fire_Points/
		2003/
		...
		2024/
		processed/
	HMS_Extent/
	Smoke_Polygons/
		2003/
		...
		2024/
		processed/
	Smoke_Text/

<b>Steps</b>:

1. Download the HMS .zip files to HMS/Smoke_Polygons/ and HMS/Fire_Points/ using ancill/HMS_Download.R and ancill/HMS_Fire_Download.R. Unzip using ancill/HMS_Unzip.R and ancill/HMS_Fire_Unzip.R.
2. Process HMS daily shapefiles and output yearly shapefiles to HMS/Smoke_Polygons/processed/ and HMS/Fire_Points/processed/ using HMS.R and HMS_Fire.R.
3. Retrieve HMS smoke text description links as .txt and output as yearly .csv tables in HMS/Smoke_Text/ using ancill/HMS_TextLinksYr.R; combine the .csv files into a single file, HMS/Smoke_Text/HMS_SmokeText.csv, using ancill/HMS_TextLinks.R

Gap-filling Unspecified Densities

We used random forest classification to assign densities (light, medium, or heavy) to polygons with unspecified densities from 2005-2010. This procedure is described in Liu et al. (2024, IJWF). Note that the code has recently been updated to use sf instead of rgdal, and additional processing has been done to fix more bad geometries. The code workflow uses EE to generate some input data for the random forest model (HMS_Stack.js,HMS_AOD.js). The rest of the workflow is in R with RFmodel_prepare.R to output a CSV table of data for all HMS polygons from 2005-2022, RFmodel_withAOD.R and RFmodel_withoutAOD.R to run the random forest classification models, RFmodel_export.R to output another CSV table now with the gap-filled densities, and finally HMS_gapfill_shp.R to rewrite HMS files from 2005-2010 with the gap-filled densities and associated flags.

Updates

• October 2024: fixed missing HMS fire points in 2007
• August 2024: replaced FIRMS with the HMS fire product for the active fires layer on the app, update MODIS burned area layer from Collection 6 to 6.1
• July 2024: added VIIRS active fires to app; there seems to be some issues with recent active fire images in the Earth Engine / FIRMS dataset
• September 2023: uploaded gap-filled HMS polygons from 2005-2010 and added related code; added ancillary code for preprocessing; added note about evaluation of the HMS smoke product on the app
• July 2023: updated R code to process HMS from rgdal to sf, code is now more inclusive of out-of-bounds polygons and attempts to fix some bad geometries (unclosed rings and out-of-bounds coordinates) but excludes polygons with crossed edges as sf cannot fix them; added QA flags; added jump to latest button
• September 2020: added option to select GOES-East or GOES-West full disk imagery, added CAMS PM_2.5 and AOD
• August 2020: added visualization of GOES RGB imagery, HMS smoke days and 'duration'

Publications

Liu, T., F.M. Panday, M.C. Caine, M. Kelp, D.C. Pendergrass, L.J. Mickley, E.A. Ellicott, M.E. Marlier, R. Ahmadov, and E.P. James (2024). Is the smoke aloft? Caveats regarding the use of the Hazard Mapping System (HMS) smoke product as a proxy for surface smoke presence across the United States. Int. J. Wildland Fire, 33, WF23148. https://doi.org/10.1071/WF23148