Home

Awesome

DiT: Efficient Vision Transformers with Dynamic Token Routing

<!-- ## Description -->

In this paper, we propose a data-dependent token routing strategy to elaborate the routing paths of image tokens for Dynamic Vision Transformer, dubbed DiT. The proposed framework generates a data-dependent path per token, adapting to the object scales and visual discrimination of tokens. In feed-forward, the differentiable routing gates are designed to select the scaling paths and feature transformation paths for image tokens, leading to multi-path feature propagation. In this way, the impact of object scales and visual discrimination of image representation can be carefully tuned. Moreover, the computational cost can be further reduced by giving budget constraints to the routing gate and early-stopping of feature extraction.

:calendar: Schedule

:dart: Motivation

<img width="800" alt="image" src="https://raw.githubusercontent.com/Maycbj/DiT/main/figures/motivation.png">

:gift: Major Features

<img width="800" alt="image" src="https://raw.githubusercontent.com/Maycbj/DiT/main/figures/method.png">

:snowman: Experiments

:eyes: Visualization

<img width="800" alt="image" src="https://raw.githubusercontent.com/Maycbj/DiT/main/figures/visualization.png">

🎫 License

This project is released under the Apache 2.0 license.