Home

Awesome

WiMANS: A Benchmark Dataset for WiFi-based Multi-user Activity Sensing

<p align="center"> <a href="#Introduction">Introduction</a> | <a href="#Environment">Environment</a> | <a href="#Dataset">Dataset</a> | <a href="#Experiments">Experiments</a> | <a href="#Citation">Citation</a> | <a href="#Contact-Us">Contact Us</a> </p>

Introduction

This repository presents WiMANS, including the dataset, source code, and documents. To the best of our knowledge, WiMANS is the first WiFi-based multi-user activity sensing dataset based on WiFi Channel State Information (CSI). WiMANS contains 11286 CSI samples of dual WiFi bands (2.4 / 5 GHz) and synchronized videos for reference and unexplored tasks (e.g., multi-user pose estimation). Each 3-second sample includes 0 to 5 users performing identical/different activities simultaneously, annotated with (anonymized) user identities, locations, and activities.

<table align = "center"> <tr align = "center"><td rowspan="2"> <b>Sample <br/> "act_30_25" </b></td><td>WiFi CSI (5GHz)</td> <td>Synchronized Video</td></tr> <tr align = "center"><td><img src="visualize/wifi_csi_act_30_25.gif" height="188"/></td><td><img src="visualize/video_act_30_25.gif" height="188"></td></tr> </table> <table align = "center"> <tr align = "center"><td rowspan="2"> <b>Sample <br/> "act_49_41"</b></td><td>WiFi CSI (2.4GHz)</td> <td>Synchronized Video</td></tr> <tr align = "center"><td><img src="visualize/wifi_csi_act_49_41.gif" height="188"/></td><td><img src="visualize/video_act_49_41.gif" height="188"/></td></tr> </table> <table align = "center"> <tr align = "center"><td rowspan="2"> <b>Sample <br/> "act_88_30" </b></td><td>WiFi CSI (2.4GHz)</td> <td>Synchronized Video</td></tr> <tr align = "center"><td><img src="visualize/wifi_csi_act_88_30.gif" height="188"/></td><td><img src="visualize/video_act_88_30.gif" height="188"/></td></tr> </table>

Environment

We suggest to create the environment by installing environment.yaml with Anaconda.

conda env create -f environment.yaml
conda activate WiMANS

Dataset

Please extract the WiMANS dataset under the dataset directory.

Annotations are saved in the "annotation.csv" file, which can be read using Pandas.

import pandas as pd
data_pd_y = pd.read_csv(var_path_data_y, dtype = str)
# "var_path_data_y" is the path of "annotation.csv"

Raw CSI data are saved in "*.mat" files, which can be read using SciPy.

import scipy.io as scio
data_mat = scio.loadmat(var_path_mat)
# "var_path_mat" is the path of "*.mat" file

The preprocessed data of CSI amplitude are saved in "*.npy" files, which can be read using NumPy.

import numpy as np
data_csi = np.load(var_path)
# "var_path" is the path of "*.npy" file

Video data are saved in "*.mp4" files, which can be read using PyTorch.

import torchvision
data_video_x, _, _ = torchvision.io.read_video(var_path, output_format = "TCHW")
# "var_path" is the path of "*.mp4" file

Experiments

The source code of benchmark experiments is under the benchmark directory.

WiFi-based Models

Data Preparation

Edit benchmark/wifi_csi/preset.py to select the number(s) of users, WiFi band(s), and environment(s).

preset = {
    ...
    "data": {
        "num_users": ["0", "1", "2", "3", "4", "5"],# e.g., ["0", "1"], ["2", "3", "4", "5"]
        "wifi_band": ["2.4"],                       # e.g., ["2.4"], ["5"], ["2.4", "5"]
        "environment": ["classroom"],               # e.g., ["classroom"], ["meeting_room"], ["empty_room"]
        ...
    },
    ...
}

(Optional) We use benchmark/wifi_csi/preprocess.py to calculate the amplitude of raw WiFi CSI. --dir_mat defines the input directory of raw WiFi CSI. --dir_amp defines the output directory of CSI amplitude. The following code uses the raw WiFi CSI in dataset/wifi_csi/mat to calculate the amplitude and saves the amplitude files to dataset/wifi_csi/amp.

python benchmark/wifi_csi/preprocess.py --dir_mat="dataset/wifi_csi/mat" --dir_amp="dataset/wifi_csi/amp"

Note that this step is optional because we have already provided the preprocessed data in dataset/wifi_csi/amp.

Run Experiments

Edit benchmark/wifi_csi/preset.py to set data directory and to tune the hyperparameters, including the learning rate, batch size, etc.

Use benchmark/wifi_csi/run.py to run experiments for WiFi-based models. --model defines the WiFi-based model for experiments. --task defines the human sensing task to evaluate. --repeat defines the number of repeated experiments. If these arguments are not provided, models will load the default settings in benchmark/wifi_csi/preset.py.

The following code repeats experiments 10 times to run the MLP model for WiFi-based human activity recognition.

python benchmark/wifi_csi/run.py --model="MLP" --task="activity" --repeat=10

After experiments, the results will be saved in result.json, which can be modified in benchmark/wifi_csi/preset.py.

Video-based Models

Data Preparation

Edit benchmark/video/preset.py to select the number(s) of users and environment(s).

preset = {
    ...
    "data": {
        "num_users": ["0", "1", "2", "3", "4", "5"],# e.g., ["0", "1"], ["2", "3", "4", "5"]
        "environment": ["classroom"],               # e.g., ["classroom"], ["meeting_room"], ["empty_room"]
    },
    ...
}

We preprocess video data according to the original papers of video-based models. --path_data_x defines the input directory of raw videos. --path_data_y defines the path of annotation file. --model defines the model for which the videos should be preprocessed. --path_data_pre_x defines the output directory of preprocessed videos (*.npy).

The following code preprocesses videos in dataset/video for the ResNet model and saves preprocessed files to dataset/cache.

python benchmark/video/preprocess.py --path_data_x="dataset/video" --path_data_y="dataset/annotation.csv" --model="ResNet" --path_data_pre_x="dataset/cache"

Run Experiments

Edit benchmark/video/preset.py to set data directory and to tune the hyperparameters, including the learning rate, batch size, etc.

Use benchmark/video/run.py to run experiments for video-based models. --model defines the video-based model for experiments. --task defines the human sensing task to evaluate. --repeat defines the number of repeated experiments. If these arguments are not provided, models will load the default settings in benchmark/video/preset.py.

The following code repeats experiments 10 times to run the ResNet model for video-based human activity recognition.

python benchmark/video/run.py --model="ResNet" --task="activity" --repeat=10

After experiments, the results will be saved in result.json, which can be modified in benchmark/video/preset.py.

Citation

If you find this repository useful for your research, please consider citing our paper:

@article{huang2024wimans,
  title={WiMANS: A Benchmark Dataset for WiFi-based Multi-user Activity Sensing},
  author={Huang, Shuokang and Li, Kaihan and You, Di and Chen, Yichong and Lin, Arvin and Liu, Siying and Li, Xiaohui and McCann, Julie A},
  journal={arXiv preprint arXiv:2402.09430},
  year={2024}
}

Contact Us

Please feel free to contact us if you have any questions about WiMANS.