Home

Awesome

InHARD - Industrial Human Action Recognition Dataset

Objectives

We introduce a RGB+S dataset named “Industrial Human Action Recognition Dataset” (InHARD) from a real-world setting for industrial human action recognition with over 2 million frames, collected from 16 distinct subjects. This dataset contains 13 different industrial action classes and over 4800 action samples. The introduction of this dataset should allow us the study and development of various learning techniques for the task of human actions analysis inside industrial environments involving human robot collaborations.

This work has been performed at the LINEACT - CESI laboratory : https://recherche.cesi.fr/inhard-industrial-human-action-recognition-dataset/

Download

The dataset can be downloaded from this link : https://zenodo.org/record/4003541
Results on this dataset can be published on Papers with code

Dataset description

dataset example

Modalities

Skeleton modality

We used a “Combination Perception Neuron 32 Edition v2” motion sensor to capture the skeletal data delivered at a frequency of 120 Hz.
Skeleton data comprises :

<p align="center"> <img src="rsc/Skeleton-joints-hierarchy.png" alt="Skeleton-joints-hierarchy"> </p>

To manipulate BVH files, we recommand using the PyMO python library

Video modality

We used 3 C920 cameras to capture RGB Data. Each camera captures three different views of the same action. For each setup, two cameras were placed at the same height but at two different horizontal angles: -45° and +45° to capture both left and right sides. The third camera is placed on top of the subjects to capture the top view.

RGB files are stored in the RGB/ folder of the InHARD dataset.

Action Classes

The list of 13 meta-actions and 74 actions classes are available in the Action-Meta-action-list.xlsx file

Inside the InHARD.zip datatset, you will find the InHARD.csv file which provides a dataframe with all dataset info including Filename, Subject, Operation, Action low/high level label, Action start/end, Duration etc. in order to facilitate the dataset handling and use.
See an extract below:

File_nameSubjectOperationAction_labelMeta_action_numberMeta_action_labelAction_start_bvh_frameAction_end_bvh_frameAction_start_rgb_secAction_end_rgb_secAction_start_rgb_frameAction_end_rgb_frameDuration_sec
P01_R01P01OP010[OP010] Consult sheets2Consult sheets32310712.729.00682256.28
P01_R01P01OP010[OP010] Catch Fixture key LARD7Picking left1342152811.2812.842823211.56
P01_R01P01OP010[OP010] Place LARD on Profile P360-112Assemble system1647224713.8418.883464725.04
P01_R01P01OP010[OP010] Catch Fixation FIXA16Picking in front2775302823.3225.445836362.12
P01_R01P01OP010[OP010] Place FIXA1 on LARD at 160mm12Assemble system3151342826.4828.806627202.32

We used a software called ANVIL to label our data. You can install it if you want to edit, add or remove actions from actions' labels files (.anvil) situated at the /Labels/ folder.

Experiments and performance metrics

We propose a set of usage metrics of the InHARD dataset for future utilization. Firstly, we suggest dividing data into two levels; experts and beginners according to subject’s expertise with the manipulation. Thereby, all subjects performing the whole manipulation in less than 6 minutes as average total actions' duration, are selected as experts. The remaining subjects are categorized as beginners. We define the training and validation sets as follows :

PS : Samples in bold are selected as Experts. The remaining are selected as beginners.

Citation

To cite this work, please use:

@INPROCEEDINGS{9209531,  
author={M. {DALLEL} and V. {HAVARD} and D. {BAUDRY} and X. {SAVATIER}},  
booktitle={2020 IEEE International Conference on Human-Machine Systems (ICHMS)},   
title={InHARD - Industrial Human Action Recognition Dataset in the Context of Industrial Collaborative Robotics},   
year={2020},  
volume={},  
number={},  
pages={1-6},}