Home

Awesome

CoastSat-Package

This is coastsat-package the pip and conda package extension of CoastSat. CoastSat-package is a slightly modified version of coastsat to make it compatible with CoastSeg. The CoastSeg team actively maintains coastsat-package and we welcome all contributions to improve the package.

Modifications and Improvements

Our recent updates to the coastsat-package were driven by the goal of enhancing functionality, user experience, and compatibility with CoastSeg. We aimed to streamline image handling, improve data accessibility, and provide more detailed feedback during the download process. Additionally, we introduced new features to allow more precise control over image processing and data collection. These enhancements also include better file organization and comprehensive shoreline extraction reports to ensure transparency and ease of use. Below is a detailed overview of these significant improvements and additions.

Install CoastSat with conda

  1. Create a conda environment
  1. Install coastsat-package using the conda forge channel

    conda install -c conda-forge coastsat-package -y

  2. (Optional) At any time you upgrade coastsat with the command:

    conda update -c conda-forge coastsat-package -y

Install CoastSat with Pip

PyPi Repo

  1. Create a conda environment
  1. Activate your conda environment

    conda activate coastsat_pkg

  2. Install jupyter and geopandas with Conda using the conda forge channel
    • Geopandas has GDAL as a dependency so its best to install it with conda.

      conda install -c conda-forge geopandas jupyterlab -y

  3. Install pip in your conda environment
  1. Install the pip package

Issues

Having a problem? Post an issue in the Issues page (please do not email).

Contributing

If you would like to contribute to the further development of coastsat-package please follow the steps below. We appreciate all contributions!

  1. Fork the repository (https://github.com/2320sharon/coastsat_package/fork). A fork is a copy on which you can make your changes.
  2. Create a new branch on your fork
  3. Commit your changes and push them to your branch
  4. When the branch is ready to be merged, create a Pull Request (how to make a clean pull request explained here)

CoastSat

Project description

Satellite remote sensing can provide low-cost long-term shoreline data capable of resolving the temporal scales of interest to coastal scientists and engineers at sites where no in-situ field measurements are available. CoastSat enables the non-expert user to extract shorelines from Landsat 5, Landsat 7, Landsat 8, Landsat 9 and Sentinel-2 images. The shoreline detection algorithm implemented in CoastSat is optimised for sandy beach coastlines. It combines a sub-pixel border segmentation and an image classification component, which refines the segmentation into four distinct categories such that the shoreline detection is specific to the sand/water interface.

The toolbox has four main functionalities:

  1. assisted retrieval from Google Earth Engine of all available satellite images spanning the user-defined region of interest and time period.
  2. automated extraction of shorelines from all the selected images using a sub-pixel resolution technique.
  3. intersection of the 2D shorelines with user-defined shore-normal transects.
  4. tidal correction using measured water levels and an estimate of the beach slope.
  5. post-processing of the shoreline time-series, despiking and seasonal averaging.
  6. validation example at Narrabeen

1. Installation

1.1 Create an environment with Anaconda

To run the toolbox you first need to install the required Python packages in an environment. To do this we will use Anaconda, which can be downloaded freely here.

Once you have it installed on your PC, open the Anaconda prompt (in Mac and Linux, open a terminal window) and use the cd command (change directory) to go the folder where you have downloaded this repository.

Create a new environment named coastsat with all the required packages by entering these commands in succession:

conda create -n coastsat python=3.8
conda activate coastsat
conda install -c conda-forge geopandas earthengine-api scikit-image matplotlib astropy notebook -y
pip install pyqt5

All the required packages have now been installed in an environment called coastsat. Always make sure that the environment is activated with:

conda activate coastsat

To confirm that you have successfully activated CoastSat, your terminal command line prompt should now start with (coastsat).

:warning: In case errors are raised :warning:: clean things up with the following command (better to have the Anaconda Prompt open as administrator) before attempting to install coastsat again:

conda clean --all

You can also install the packages with the Anaconda Navigator, in the Environments tab. For more details, the following link shows how to create and manage an environment with Anaconda.

1.2 Activate Google Earth Engine Python API

First, you need to request access to Google Earth Engine at https://signup.earthengine.google.com/. It takes about 1 day for Google to approve requests.

Once your request has been approved, with the coastsat environment activated, run the following command on the Anaconda Prompt to link your environment to the GEE server:

earthengine authenticate

A web browser will open, login with a gmail account and accept the terms and conditions. Then copy the authorization code into the Anaconda terminal. In the latest version of the earthengine-api, the authentication is done with gcloud. If an error is raised about gcloud missing, go to https://cloud.google.com/sdk/docs/install and install gcloud. After you have installed it, close the Anaconda Prompt and restart it, then activate the environenment before running earthengine authenticate again.

Now you are ready to start using the CoastSat toolbox!

Note: remember to always activate the environment with conda activate coastsat each time you are preparing to use the toolbox.

2. Usage

An example of how to run the software in a Jupyter Notebook is provided in the repository (example_jupyter.ipynb). To run this, first activate your coastsat environment with conda activate coastsat (if not already active), and then type:

jupyter lab

A web browser window will open. Point to the directory where you downloaded this repository and click on example_jupyter.ipynb.

The following sections guide the reader through the different functionalities of CoastSat with an example at Narrabeen-Collaroy beach (Australia). If you prefer to use Spyder, PyCharm or other integrated development environments (IDEs), a Python script named example.py is also included in the repository.

If using example.py on Spyder, make sure that the Graphics Backend is set to Automatic and not Inline (as this mode doesn't allow to interact with the figures). To change this setting go under Preferences>IPython console>Graphics.

A Jupyter Notebook combines formatted text and code. To run the code, place your cursor inside one of the code sections and click on the run cell button (or press Shift + Enter) and progress forward.

image

2.1 Retrieval of the satellite images

To retrieve from the GEE server the available satellite images cropped around the user-defined region of coastline for the particular time period of interest, the following variables are required:

The call metadata = SDS_download.retrieve_images(inputs) will launch the retrieval of the images and store them as .TIF files (under /filepath/sitename). The metadata contains the exact time of acquisition (in UTC time) of each image, its projection and its geometric accuracy. If the images have already been downloaded previously and the user only wants to run the shoreline detection, the metadata can be loaded directly by running metadata = SDS_download.get_metadata(inputs).

The screenshot below shows an example of inputs that will retrieve all the images of Collaroy-Narrabeen (Australia) acquired by Sentinel-2 in December 2017.

doc1

Note: The area of the polygon should not exceed 100 km2, so for very long beaches split it into multiple smaller polygons.

2.2 Shoreline detection

To map the shorelines, the following user-defined settings are needed:

There are additional parameters (min_beach_size, buffer_size, min_length_sl, cloud_mask_issue, sand_color and pan_off that can be tuned to optimise the shoreline detection (for Advanced users only). For the moment leave these parameters set to their default values, we will see later how they can be modified.

An example of settings is provided here:

image

Once all the settings have been defined, the batch shoreline detection can be launched by calling:

output = SDS_shoreline.extract_shorelines(metadata, settings)

When check_detection is set to True, a figure like the one below appears and asks the user to manually accept/reject each detection by pressing on the keyboard the right arrow (⇨) to keep the shoreline or left arrow (⇦) to skip the mapped shoreline. The user can break the loop at any time by pressing escape (nothing will be saved though).

map_shorelines

When adjust_detection is set to True, a figure like the one below appears and the user can adjust the position of the shoreline by clicking on the histogram of MNDWI pixel intensities. Once the threshold has been adjusted, press Enter and then accept/reject the image with the keyboard arrows.

Alt text

Once all the shorelines have been mapped, the output is available in two different formats (saved under /filepath/data/sitename):

The figure below shows how the satellite-derived shorelines can be opened in a GIS software (QGIS) using the .geojson output. Note that the coordinates in the .geojson file are in the spatial reference system defined by the output_epsg.

<p align="center"> <img width="500" height="300" src="https://user-images.githubusercontent.com/7217258/49361401-15bd0480-f730-11e8-88a8-a127f87ca64a.jpeg"> </p>

Reference shoreline

Before running the batch shoreline detection, there is the option to manually digitize a reference shoreline on one cloud-free image. This reference shoreline helps to reject outliers and false detections when mapping shorelines as it only considers as valid shorelines the points that are within a defined distance from this reference shoreline. It is highly recommended to use a reference shoreline.

The user can manually digitize one or several reference shorelines on one of the images by calling:

settings['reference_shoreline'] = SDS_preprocess.get_reference_sl_manual(metadata, settings)
settings['max_dist_ref'] = 100 # max distance (in meters) allowed from the reference shoreline

This function allows the user to click points along the shoreline on cloud-free satellite images, as shown in the animation below.

ref_shoreline

The maximum distance (in metres) allowed from the reference shoreline is defined by the parameter max_dist_ref. This parameter is set to a default value of 100 m. If you think that 100 m buffer from the reference shoreline will not capture the shoreline variability at your site, increase the value of this parameter. This may be the case for large nourishments or eroding/accreting coastlines.

Advanced shoreline detection parameters

As mentioned above, there are some additional parameters that can be modified to optimise the shoreline detection:

Re-training the classifier

CoastSat's shoreline mapping alogorithm uses an image classification scheme to label each pixel into 4 classes: sand, water, white-water and other land features. While this classifier has been trained using a wide range of different beaches, it may be that it does not perform very well at specific sites that it has never seen before. You can train a new classifier with site-specific training data in a few minutes by following the Jupyter notebook in re-train CoastSat classifier.

2.3 Shoreline change analysis

This section shows how to obtain time-series of shoreline change along shore-normal transects. Each transect is defined by two points, its origin and a second point that defines its length and orientation. The origin is always defined first and located landwards, the second point is located seawards. There are 3 options to define the coordinates of the transects:

  1. Interactively draw shore-normal transects along the mapped shorelines:
transects = SDS_transects.draw_transects(output, settings)
  1. Load the transect coordinates from a .geojson file:
transects = SDS_tools.transects_from_geojson(path_to_geojson_file)
  1. Create the transects by manually providing the coordinates of two points:
transects = dict([])
transects['Transect 1'] = np.array([[342836, ,6269215], [343315, 6269071]])
transects['Transect 2'] = np.array([[342482, 6268466], [342958, 6268310]])
transects['Transect 3'] = np.array([[342185, 6267650], [342685, 6267641]])

Note: if you choose option 2 or 3, make sure that the points that you are providing are in the spatial reference system defined by settings['output_epsg'].

Once the shore-normal transects have been defined, the intersection between the 2D shorelines and the transects is computed with the following function:

settings['along_dist'] = 25
cross_distance = SDS_transects.compute_intersection(output, transects, settings)

The parameter along_dist defines the along-shore distance around the transect over which shoreline points are selected to compute the intersection. The default value is 25 m, which means that the intersection is computed as the median of the points located within 25 m of the transect (50 m alongshore-median). This helps to smooth out localised water levels in the swash zone.

An example is shown in the animation below:

transects

There is also the option to run SDS_transects.compute_intersection_QA(), this function provides more quality-control when computing the intersections between shorelines and transects (small loops, multiple intersections etc).

It is recommended to use this function as it can provide cleaner shoreline time-series. See the Jupyter Notebook for a detailed description of the parameters. An example of parameters for the quality control are provided below:

image

2.4 Tidal Correction

Each satellite image is captured at a different stage of the tide, therefore a tidal correction is necessary to remove the apparent shoreline changes cause by tidal fluctuations.

In order to tidally-correct the time-series of shoreline change you will need the following data:

Wave setup and runup corrections are not included in the toolbox, but for more information on these additional corrections see Castelle et al. 2021.

2.5 Post-processing

The tidally-corrected time-series can be post-processed to remove outliers with a despiking algorithm in SDS_transects.reject_outliers().

image

Functions to compute seasonal and monthly averages on the shoreline time-series are also provided: SDS_transects.seasonal_averages() and SDS_transects.monthly_averages().

NA1

2.6 Validation against survey data

This section provides code to compare the satellite-derived shorelines against the survey data for Narrabeen, available at http://narrabeen.wrl.unsw.edu.au/.

comparison_transect_PF1

References

  1. Vos K., Splinter K.D., Harley M.D., Simmons J.A., Turner I.L. (2019). CoastSat: a Google Earth Engine-enabled Python toolkit to extract shorelines from publicly available satellite imagery. Environmental Modelling and Software. 122, 104528. https://doi.org/10.1016/j.envsoft.2019.104528 (Open Access)

  2. Vos K., Harley M.D., Splinter K.D., Simmons J.A., Turner I.L. (2019). Sub-annual to multi-decadal shoreline variability from publicly available satellite imagery. Coastal Engineering. 150, 160–174. https://doi.org/10.1016/j.coastaleng.2019.04.004

  3. Vos K., Harley M.D., Splinter K.D., Walker A., Turner I.L. (2020). Beach slopes from satellite-derived shorelines. Geophysical Research Letters. 47(14). https://doi.org/10.1029/2020GL088365 (Open Access preprint here)

  4. Castelle B., Masselink G., Scott T., Stokes C., Konstantinou A., Marieu V., Bujan S. (2021). Satellite-derived shoreline detection at a high-energy meso-macrotidal beach. Geomorphology. volume 383, 107707. https://doi.org/10.1016/j.geomorph.2021.107707

  5. Vos, K. and Deng, W. and Harley, M. D. and Turner, I. L. and Splinter, K. D. M. (2022). Beach-face slope dataset for Australia. Earth System Science Data. volume 14, 3, p. 1345--1357. https://doi.org/10.5194/essd-14-1345-2022

  6. Training dataset used for pixel-wise classification in CoastSat: https://doi.org/10.5281/zenodo.3334147