Home

Awesome

Semantic Segmentation Editor

A web based labeling tool for creating AI training data sets (2D and 3D). The tool has been developed in the context of autonomous driving research. It supports images (.jpg or .png) and point clouds (.pcd). It is a Meteor app developed with React, Paper.js and three.js.

Latest changes

Bitmap Image Editor

:movie_camera: VIDEO: Bitmap labeling overview

<a href="https://github.com/dmandrioli/sse-extra/raw/master/Capture2D1.PNG"><img width="400" src="https://github.com/dmandrioli/sse-extra/raw/master/Capture2D1.jpg"/></a> <a href="https://github.com/dmandrioli/sse-extra/raw/master/Capture2D2.PNG"><img width="400" src="https://github.com/dmandrioli/sse-extra/raw/master/Capture2D2.jpg"/></a>

PCD Point Cloud Editor

:movie_camera: VIDEO: Point cloud labeling overview

<a href="https://github.com/dmandrioli/sse-extra/raw/master/Capture3D1.PNG"><img width="400" src="https://github.com/dmandrioli/sse-extra/raw/master/Capture3D1.jpg"/></a> <a href="https://github.com/dmandrioli/sse-extra/raw/master/Capture3D2.PNG"><img width="400" src="https://github.com/dmandrioli/sse-extra/raw/master/Capture3D2.jpg"/></a>

How to run

Using Docker Compose

  1. Download the docker compose stack file (sse-docker-stack.yml)
  2. Set the folder that contains bitmap and point cloud files (YOUR_IMAGES_PATH) and run the tool using docker-compose
  3. The tool runs by default on port 80, you can change the mapping in sse-docker-stack.yml
wget https://raw.githubusercontent.com/Hitachi-Automotive-And-Industry-Lab/semantic-segmentation-editor/master/sse-docker-stack.yml
wget https://raw.githubusercontent.com/Hitachi-Automotive-And-Industry-Lab/semantic-segmentation-editor/master/settings.json
METEOR_SETTINGS=$(cat ./settings.json) SSE_IMAGES=YOUR_IMAGES_PATH docker-compose -f stack.yml up

(Optional) You can modify settings.json to customize classes data.

Running from source

Install Meteor (OSX or Linux)

curl https://install.meteor.com/ | sh

or download Meteor Windows Installer

Download and unzip latest version from here

Start the application

cd semantic-segmentation-editor-x.x.x
meteor npm install
meteor npm start

The editor will run by default on http://localhost:3000

(Optional) Edit settings.json

By default, images are served from <code>your_home_dir/sse-images</code> and pointcloud binary segmentation data are stored in <code>your_home_dir/sse-internal</code>. You can configure these folders in settings.json by modifying <code>images-folder</code> and <code>internal-folder</code> properties. On Windows, use '/' separators, example <code>c:/Users/john/images</code>

Check Meteor Environment Variables to configure your app (MONGO_URL, DISABLE_WEBSOCKETS, etc...)

Configuration File: settings.json

{
  "configuration": {
    "images-folder": "/mnt/images", // The root folder containing images and PCD files
    "internal-folder": "/mnt/pointcloud_data" // Segmentation data (only 3D) will be stored in this folder
  },
  // The different sets of classes available in the tool
  // For object classes, only the 'label' field is mandatory
  // The icon field can be set with an icon from the mdi-material-ui package
  "sets-of-classes": [
    {
      "name": "Cityscapes", "objects": [
      {"label": "VOID", "color": "#CFCFCF"},
      {"label": "Road", "color": "#804080", "icon": "Road"},
      {"label": "Sidewalk", "color": "#F423E8", "icon": "NaturePeople"},
      {"label": "Parking", "color": "#FAAAA0", "icon": "Parking"},
      {"label": "Rail Track", "color": "#E6968C", "icon": "Train"},
      {"label": "Person", "color": "#DC143C", "icon": "Walk"},
      {"label": "Rider", "color": "#FF0000", "icon": "Motorbike"},
      {"label": "Car", "color": "#0000E8", "icon": "Car"}
    },
    { ... }
  ]
}

How to use

The editor is built around 3 different screens:

The file navigator let's you browse available files to select a bitmap images or a point cloud for labeling <img width="300" src="https://github.com/dmandrioli/sse-extra/raw/master/CaptureN1.jpg"/>

The bitmap image editor is dedicated to the labeling of jpg and png files by drawing polygons <img width="300" src="https://github.com/dmandrioli/sse-extra/raw/master/Capture2D2.jpg"/>

The point cloud editor is dedicated to the labeling of point clouds by creating objects made of subsets of 3D points <img width="300" src="https://github.com/dmandrioli/sse-extra/raw/master/Capture3D3.jpg"/>

Using the bitmap image editor

There are several tools to create labeling polygons:

Polygon Drawing Tool (P)

Magic Tool (A)

Manipulation Tool (Alt)

Cutting/Expanding Tool (C)

Contiguous Polygon Tool (F)

Using the point cloud editor

PCD support

API Endpoints