Home

Awesome

📒Awesome-SD-Inference

<div align='left'> <img src=https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg > <img src=https://img.shields.io/github/forks/DefTruth/Awesome-SD-Inference.svg?style=social > <img src=https://img.shields.io/github/stars/DefTruth/Awesome-SD-Inference.svg?style=social > <img src=https://img.shields.io/github/watchers/DefTruth/Awesome-SD-Inference.svg?style=social > <img src=https://img.shields.io/badge/Release-v0.5-brightgreen.svg > <img src=https://img.shields.io/badge/License-GPLv3.0-turquoise.svg > </div>

📒A small curated list of Awesome SD/DiT/Diffusion Inference with Distributed/Caching/Sampling. For Awesome LLM Inference, please check 📖Awesome-LLM-Inference

🤖Contents

©️Citations

@misc{Awesome-SD-Inference@2024,
  title={Awesome-SD-Inference: A small curated list of Awesome SD/DiT/ViT/Diffusion Inference with Distributed/Caching/Sampling.},
  url={https://github.com/DefTruth/Awesome-SD-Inference},
  note={Open-source software available at https://github.com/DefTruth/Awesome-SD-Inference},
  author={DefTruth},
  year={2024}
}

📙Awesome SD Inference with Sampling

<div id="Sampling"></div>
DateTitlePaperCodeRecom
2020.06🔥[DDPM] Denoising Diffusion Probabilistic Models(@UC Berkeley)[pdf][diffusion] ⭐️⭐️
2020.10🔥[DDIM] DENOISING DIFFUSION IMPLICIT MODELS(@cs.stanford.edu)[pdf]⚠️⭐️⭐️
2022.02🔥[PNDM] PSEUDO NUMERICAL METHODS FOR DIFFUSION MODELS ON MANIFOLDS(@)[pdf][PNDM] ⭐️⭐️
2022.02🔥[DPM-Solver] DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps(@Cheng Lu)[pdf][dpm-solver] ⭐️⭐️
2022.11🔥[DPM-Solver++] DPM-SOLVER++: FAST SOLVER FOR GUIDED SAMPLING OF DIFFUSION PROBABILISTIC MODELS(@Cheng Lu)[pdf][dpm-solver] ⭐️⭐️
2023.10🔥[DPM-Solver-v3] DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics(@Kaiwen Zheng)[pdf][DPM-Solver-v3] ⭐️⭐️
2023.11🔥[Parallel Sampling] Parallel Sampling of Diffusion Models(@Stanford University)[pdf][paradigms] ⭐️⭐️
2023.11🔥[SAMPLER SCHEDULER] SAMPLER SCHEDULER FOR DIFFUSION MODELS(@sysu)[pdf]⚠️⭐️⭐️
2024.02🔥[Parallel Sampling] Accelerating Parallel Sampling of Diffusion Models(@Zhiwei Tang)[pdf][ParaTAA-Diffusion] ⭐️⭐️
2024.01🔥[YONOS] You Only Need One Step: Fast Super-Resolution with Stable Diffusion via Scale Distillation(@Samsung AI)[pdf]⚠️⭐️⭐️
2024.01🔥[S^2-DM] S^2-DMs: Skip-Step Diffusion Models(@Yixuan Wang)[pdf]⚠️⭐️⭐️
2024.08🔥[StepSaver] StepSaver: Predicting Minimum Denoising Steps for Diffusion Model Image Generation(@intel)[pdf]⚠️⭐️⭐️
2024.09🔥[DC-Solver] DC-Solver: Improving Predictor-Corrector Diffusion Sampler via Dynamic Compensation(@Tsinghua University)[pdf][DC-Solver] ⭐️⭐️

📙Awesome SD Inference with Caching

<div id="Caching"></div> <img width="1645" alt="image" src="https://github.com/user-attachments/assets/a7257462-80d3-40af-a4ce-3550508fabe7">
DateTitlePaperCodeRecom
2023.05🔥🔥[Cache-Enabled Sparse Diffusion] Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference(@pku.edu.cn etc)[pdf]⚠️⭐️⭐️
2023.12🔥🔥[DeepCache] DeepCache: Accelerating Diffusion Models for Free(@nus.edu)[pdf][DeepCache] ⭐️⭐️
2023.12🔥🔥[Block Caching] Cache Me if You Can: Accelerating Diffusion Models through Block Caching(@Meta GenAI etc)[pdf]⚠️⭐️⭐️
2023.12🔥🔥[Approximate Caching] Approximate Caching for Efficiently Serving Diffusion Models(@Adobe)[pdf]⚠️⭐️⭐️
2024.06🔥🔥[Layer Caching] Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching(@nus.edu)[pdf][learning-to-cache] ⭐️⭐️
2024.07🔥[ElasticCache-LVLM] Efficient Inference of Vision Instruction-Following Models with Elastic Cache(@Tsinghua University etc)[pdf][ElasticCache] ⭐️
2024.07🔥🔥[Fast-Forward Caching(DiT)] FORA: Fast-Forward Caching in Diffusion Transformer Acceleration(@microsoft.com etc)[pdf][FORA] ⭐️⭐️
2024.07🔥🔥[Faster I2V Generation] Faster Image2Video Generation: A Closer Look at CLIP Image Embedding’s Impact on Spatio-Temporal Cross-Attentions(@Ashkan Taghipour etc)[pdf]⚠️⭐️⭐️
2024.04🔥🔥[T-GATE V1] Cross-Attention Makes Inference Cumbersome in Text-to-Image Diffusion Models(@Wentian Zhang etc)[pdf][T-GATE] ⭐️⭐️
2024.04🔥🔥[T-GATE V2] Faster Diffusion via Temporal Attention Decomposition(@Haozhe Liu etc)[pdf][T-GATE] ⭐️⭐️

📙Awesome SD Distributed Inference with Multi-GPUs

<div id="Distributed"></div> <img width="1677" alt="image" src="https://github.com/user-attachments/assets/aefb2ae7-73eb-4e9c-bf1a-ec540f4dfa7d"> <img width="1346" alt="image" src="https://github.com/user-attachments/assets/692c5d54-19b3-4ce7-9613-9eb8bb035c7d">
DateTitlePaperCodeRecom
2024.02🔥🔥[DistriFusion] DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models(@MIT etc)[pdf][distrifuser] ⭐️⭐️
2024.05🔥🔥[PipeFusion] PipeFusion: Displaced Patch Pipeline Parallelism for Inference of Diffusion Transformer Models(@Tencent etc)[pdf][xDiT] ⭐️⭐️
2024.06🔥🔥[AsyncDiff] AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising(@nus.edu)[pdf][AsyncDiff] ⭐️⭐️
2024.05🔥🔥[TensorRT-LLM SDXL] SDXL Distributed Inference with TensorRT-LLM and synchronous comm(@Zars19)[pdf][SDXL-TensorRT-LLM] ⭐️⭐️
2024.06🔥🔥[Clip Parallelism] Video-Infinity: Distributed Long Video Generation(@nus.edu)[pdf][Video-Infinity] ⭐️⭐️
2024.05🔥🔥[FIFO-Diffusion] FIFO-Diffusion: Generating Infinite Videos from Text without Training(@Seoul National University)[pdf][FIFO-Diffusion] ⭐️⭐️

📙Other Awesome SD Inference Paper with codes

<div id="Others"></div>
DateTitlePaperCodeRecom
2024.06🔥[DiTFastAttn] DiTFastAttn: Attention Compression for Diffusion Transformer Models(@Zhihang Yuan etc)[pdf][DiTFastAttn] ⭐️⭐️
2024.08🔥[Transfusion] Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model(@meta)[pdf][transfusion-pytorch] ⭐️⭐️
2024.08🔥[VQ4DiT] VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers(@ZJU)[pdf]⚠️⭐️⭐️
2024.08🔥[LBQ] Low-Bitwidth Floating Point Quantization for Efficient High-Quality Diffusion Models(@toronto.edu)[pdf]⚠️⭐️⭐️
2024.08🔥[EE-Diffusion] A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models(@KAIST AI)[pdf][ee-diffusion] ⭐️⭐️
2024.08🔥[TFM-PTQ] Temporal Feature Matters: A Framework for Diffusion Model Quantization(@SenseTime)[pdf]⚠️⭐️⭐️
2024.08🔥[Diffusion-RWKV] Diffusion-RWKV: Scaling RWKV-Like Architectures for Diffusion Models(@Zhengcong Fei)[pdf][Diffusion-RWKV] ⭐️⭐️
2024.09🔥[LinFusion] LINFUSION: 1 GPU, 1 MINUTE, 16K IMAGE(@NUS)[pdf][LinFusion] ⭐️⭐️

©️License

GNU General Public License v3.0

🎉Contribute

Welcome to star & submit a PR to this repo!

<div align='center'> <a href="https://star-history.com/#DefTruth/Awesome-SD-Inference&Date"> <picture align='center'> <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=DefTruth/Awesome-SD-Inference&type=Date&theme=dark" /> <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=DefTruth/Awesome-SD-Inference&type=Date" /> <img width="350" height="250" alt="Star History Chart" src="https://api.star-history.com/svg?repos=DefTruth/Awesome-SD-Inference&type=Date" /> </picture> </a> </div>