Home

Awesome

3ET is an event-based eye-tracking dataset to facilitate the download! image πŸ’₯πŸ’₯πŸ’₯ Loading 3ET dataset is now available with the Tonic event datasets loader Jump to dataloader The Tonic data loader helps you to easily load raw events which can be further processed into your favorite event data representation including event frames. A eye-tracking game demo developed at Telluride workshop 2023,CO, US: https://www.youtube.com/watch?v=aRB5mDNfrHM

If you find this repo helpful, please cite our paper. This is the archived version and the BioCAS proceeding version will follow.

@INPROCEEDINGS{3et,
  author={Chen, Qinyu and Wang, Zuowen and Liu, Shih-Chii and Gao, Chang},
  booktitle={2023 IEEE Biomedical Circuits and Systems Conference (BioCAS)}, 
  title={3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network}, 
  year={2023},
  pages={1-5},
  doi={10.1109/BioCAS58349.2023.10389062}}

3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network

This repo introduces you to how to perform pupil detection using event stream from event-based cameras. Here is our paper: 3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network, which was published at BioCAS 2023. The arxiv version: https://arxiv.org/pdf/2308.11771.pdf

Synthetic Event-based Eye Tracking dataset link (event frame): Google Drive

Synthetic Event-based Eye Tracking dataset link (event raw data and .avi video files): Google Drive

Run steps:

  1. Download the SEET dataset, and save it in a directory. i.e. /DATA/

  2. cd eyetracking-convlstm

  3. run process_event.py

    you can change the sequence length by setting the parameter seq

  4. run convlstm-et-pytorch-event.py

Results: x, y coordinates of pupil center predictions after 28 epochs of training ζˆ‘ηš„ε›Ύη‰‡ζθΏ°

Original LPW dataset (not event-based dataset)： https://www.mpi-inf.mpg.de/departments/computer-vision-and-machine-learning/research/gaze-based-human-computer-interaction/labelled-pupils-in-the-wild-lpw

<a id="tonic-dataloader"></a>

Loading raw 3ET with the Tonic library πŸ’₯πŸ’₯πŸ’₯

We support using the Tonic library to automatically download the 3ET dataset and load the raw events and labels.

The Tonic library is a Python library for loading and processing event-based data. It is available at GitHub Repo

To install the Tonic library branch which includes the 3ET dataset loader, run the following command:

pip install tonic --pre

A minimum working example of loading the 3ET dataset is shown below:

import tonic

trainset = tonic.datasets.ThreeET_Eyetracking(save_to="./data", split='Train')

events, targets = next(iter(trainset))