Awesome

MA-VIED: A Multisensor Automotive Visual Inertial Event Dataset

MA-VIED is a real-world extensive set of data to support the development of solutions and algorithms for sensor fusion, event-based and frame-based visual tracking, including visual, visual-inertial odometry and SLAM applications. </br>

<img src="imgs\Front.png" alt="MA-VIED Automotive Dataset" width="850px"/> <br/>

The MA-VIED Dataset is intended for

• evaluating the performance of vision algorithms in the context of Visual Inertial Odometry (VIO) and Simultaneous Localization and Mapping (SLAM) tasks.
• supporting research endeavors that leverage large-scale driving datasets, especially those equipped with novel neuromorphic vision sensors.

License

MA-VIED is released under the following License:

Attribution-NonCommercial-ShareAlike 3.0 CC BY-NC-SA 3.0.

This means it is possible:

• to copy, distribute, display, and perform the work.
• to make derivative works.

Under the following conditions:</br>

• Attribution: You must give the original author credit.</br>
• Non-Commercial — You may not use this work for commercial purposes. </br>
• Share Alike — If you alter, transform, or build upon this work, you may distribute the resulting work only under a licence identical to this one.

Privacy Disclaimer

Notice: This automotive dataset is provided as-is and may contain personal information. We take user privacy seriously and have implemented measures to minimize the inclusion of sensitive data. However, given the complexity of real-world data, it is possible that some personal information may be present.

User Notification and Data Removal:

Users are encouraged to review the dataset carefully. If you identify any personal data that should not be included, please promptly raise an issue on our GitHub repository. We are committed to promptly addressing and resolving privacy concerns. Upon receiving a valid request, we will make reasonable efforts to remove or anonymize the specified personal data from the dataset.

Important Considerations:

The dataset is intended for research and educational purposes. Users are responsible for complying with privacy and data protection regulations when using this dataset. We do not assume liability for any unauthorized use of the dataset.

Contact Information:

For privacy-related concerns or data removal requests, please contact us at giuseppe.mollica@dottorandi.unipg.it

Citing:

If you use MA-VIED in an academic work, please cite:

@article{10254473,
  author={Mollica, Giuseppe and Felicioni, Simone and Legittimo, Marco and Meli, Leonardo and Costante, Gabriele and Valigi, Paolo},
  journal={IEEE Transactions on Intelligent Transportation Systems}, 
  title={MA-VIED: A Multisensor Automotive Visual Inertial Event Dataset}, 
  year={2024},
  volume={25},
  number={1},
  pages={214-224},
  keywords={Cameras;Sensors;Wheels;Sensor fusion;Odometry;Standards;Visualization;Visual inertial odometry;event vision;MA-VIED automotive dataset;sensor fusion},
  doi={10.1109/TITS.2023.3312355}
}

1. The sensor setup

We compiled a dataset using a sensor setup installed on the roof of an electric KIA Soul. Below, we outline the vehicle specifications and the sensor equipment employed. The setup includes Two Wheel Pulse Transducers (WPT) connected to both the left and right wheels of the vehicle. Additionally, the inertial-vision (IV) system comprises an event camera, a standard camera, and an IMU. The in-car sensor data can be accessed through the OBDII interface.

1.1 Sensors, Data Synchronization and Triggering Mechanism

All sensor data is synchronized using a trigger signal dispatched by a microcontroller, ensuring precise timestamps for accurate temporal alignment.

The microcontroller computes precise millisecond timestamps for each IMU measurement at a frequency of 200Hz. At specific timestamps (28Hz), it initiates various actions:

• Triggers the camera (via the trigger line) to capture a new image.
• Instructs the event camera to record the trigger-in as event data at a rate of 100Hz.
• Commands both RTK receivers (at 28Hz) to log the input trigger.

Timestamps and trigger sequence numbers are available through three distinct ROS topics. Given the varied requirements for alignment, we chose to retain the triggering information, allowing users the flexibility to implement post-processing and alignment procedures as needed.

The 3D CAD model for the roof-mounted setup can be found in the 'additional_material' folder on this project's page.

Note: Due to the unavailability of trigger input, the CAN and wheel pulse transducer data are not hardware synchronized with other sensors.

Note: Prophesee EVKs and sensors support external triggering (input or output signals), enabling control of the sensor and data synchronization across multiple devices. For details on the event camera triggering mechanism, please refer to the <a href="https://docs.prophesee.ai/stable/hw/manuals/timing_interfaces.html#trigger-in-and-external-trigger-events">Prophesee documentation</a>.

2. Data collection campaign

MA-VIED was collected using a Kia Soul electric vehicle equipped with diverse heterogeneous sensors, capturing crucial information for vehicle localization and tracking. The dataset comprises sequences recorded during 13 experimental sessions in varied driving environments (highway, country road, city, racetrack-like) and styles (smooth and rapid maneuvers). These sessions took place in December 2021, as the electric vehicle navigated for several kilometers through different streets in Passignano sul Trasimeno (PG, Umbria, Italy).

<img src="imgs\all_traj_sep.png" alt="MA-VIED Sample Sequences" width="870px"/> <br/>

The accompanying table illustrates the percentage of RTK fixed data in relation to the total GPS data for both RTK antennas.

Note: The RTK GPS provides data in three modes, ranked by increasing accuracy: Autonomous, RTK Float, and RTK Fix. The RTK Fix mode is the most precise, with a deviation of 1-6cm. However, it is important to note that the high sensitivity of the RTK link makes it susceptible to signal loss in the presence of elements such as trees or buildings, which can result in reduced positioning accuracy. Across various sequences, the RTK evaluation frequently oscillates between fixed and float statuses.

2.1 Dataset

The released sequences have been meticulously curated to ensure high-quality data and variability, particularly in terms of lighting conditions and texturing. Additionally, the sequences encompass diverse use cases, including event-based autonomous parking, race tracking for virtual coach applications, and various robotic challenges such as Visual Odometry (VO), Visual-Inertial Odometry (VIO), and Simultaneous Localization and Mapping (SLAM).

In addition to the evaluation sequences, we are providing our setup CAD model and calibration sequences. This enables users to conduct their calibration, even though we also provide the calibration results.

LINK TO DATASET DOWNLOAD

http://sira.diei.unipg.it/supplementary/public/Datasets/MA-VIED/Data/

3 ROSBAG for onboard devices recorded data.

We used ROS Melodic, utilizing the ROSBAG structure and the broader ROS infrastructure to collect and manage all data. </br>

The rosbag package facilitates the recording of various topics and messages into a single file, enabling batch execution for experiment reproduction. The subsequent table provides detailed information about the collected data, including message types and available topics within this dataset.

Note: for all piksi multi data, 0 refers to front RTK antenna and 1 refers to rear RTK antenna

3.1 Custom ROS topics

3.1.1 /KIA_SOUL_EV/ART/car_state topic

The KIA_SOUL_EV/ART/car_state topic contains information related to wheel ticks (utilizing Kistler wheel encoders), steering angle (vehicle CAN), current gear position (vehicle CAN), and brake pedal pressure (expressed in the range 0-1 and extracted from vehicle CAN). Specifically:

• encoder_pulses_RR: Count of ticks for the right wheel.
• encoder_pulses_RL: Count of ticks for the left wheel.
• steering_angle: Steering wheel angle expressed in degrees.
• actual_gear: Defines the current gear according to the following numbering:
  • NO_COMMAND: 0
  • NEUTRAL: 1
  • FORWARD: 2
  • REVERSE: 3
  • EMERGENCY_STOP: 4
• brake_pressure: Brake pedal pressure status (0 for brake completey released and 1 for brake completely pressed).

3.1.2 /can_frame_can0 topic

This topic encompasses the RAW unparsed data retrieved from the Can vehicle interface.

3.1.3 /piksi_ttyUSB/ topic

Piksi RTK custom messages. For additional information, please refer to https://github.com/ethz-asl/ethz_piksi_ros.

4 Calibration

We provide the following calibration parameters for cameras and IMU sensors:

• Intrinsic camera matrix for event and frame cameras,
• IMU biases and noise,
• Extrinsic parameters: event camera to IMU, camera frame to IMU, and RTK to IMU parameters for both RTK antennas (derived from the 3D CAD model).

In addition to the evaluation sequences, our release includes the setup CAD model and calibration sequences. This empowers users to perform their calibration, even though we do provide the calibration results.

The following sequences are available:

The checkerboards used for camera and camera-to-IMU calibrations are as follows:

LINK TO CALIBRATION DATASET DOWNLOAD

http://sira.diei.unipg.it/supplementary/public/Datasets/MA-VIED/Calibration/

4.1 Extrinsic Calibration

4.1.1 Imu -> Basler Extrinsic Parameters

4.1.2 Imu -> Prophesee Extrinsic Parameters

4.2 Intrisic Calibration

4.2.1 Basler Dart Pinhole Model Calibration Parameters

4.2.2 Prophesee Pinhole Model Calibration Parameters

4.2.3 IMU Xsens G-710 Calibration Parameters

Partners

This project is a collaborative effort between ART S.p.A and the University of Perugia, Faculty of Engineering. The collaboration brings together industry expertise and academic research to contribute to the development of this automotive dataset.

Collaborators: