Home

Awesome

SIFR Dataset and QAGNet

This is the official repository for the CVPR 2024 paper Advancing Saliency Ranking with Human Fixations: Dataset, Models and Benchmarks. [Paper Link]

SIFR Dataset - From Mouse-Trajectories to Real Human Gaze

We present the first saliency ranking dataset, SIFR, using genuine human fixations rather than mouse movements (ASSR and IRSR).

Differences between Human Gaze GT and Mouse-Trajectory GT:

Example Image

Dataset Analysis:

Example Image

Download SIFR dataset (Google Drive).

QAGNet - Query as Graph Network

To establish a baseline for this dataset, we propose QAGNet, a novel model that leverages salient instance query features from a query-based transformer detector (Mask2Former) within a tri-tiered nested graph.

Example Image

Installation

Our proposed QAGNet is based on Mask2Former. Please follow the instructions to install the environment (Pytorch, Detectron2, MSDeformAttn etc). The main codes of QAGNet are written in ./mask2former/modeling/transformer_decoder/mask2former_transformer_decoder.py.

Training Model

Testing QAGNet

Benchmark

We opensource all SRD models' predicted saliency ranking maps (Google Drive). For more quantitative comparison and qualitative comparisons, please refer to our our paper and supplementary material.

Example Image

FAQ

1. ImportError: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found

Check if you can load cuda-11.1/cudnn-v8.1.1.33/gcc-10.2.0

2. LooseVersion = distutils.version.LooseVersionAttributeError: module 'distutils' has no attribute 'version'

Please install setuptools 59.5.0

pip install setuptools==59.5.0

Citing SIFR Dataset and QAGNet

If you find the SIFR dataset or QAGNet beneficial for your work, please consider citing our research:

@inproceedings{deng2024advancing,
  title={Advancing Saliency Ranking with Human Fixations: Dataset Models and Benchmarks},
  author={Deng, Bowen and Song, Siyang and French, Andrew P and Schluppeck, Denis and Pound, Michael P},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={28348--28357},
  year={2024}
}