Home

Awesome

SelfSup

Collections of self-supervised methods (MoCo series, SimCLR, SiMo, BYOL, SimSiam, SwAV, PointContrast, etc.).

Get Started

Install cvpods following the instructions.

Install cvpods from https://github.com/Megvii-BaseDetection/cvpods.git .

Prepare Datasets

cd cvpods
ln -s /path/to/your/ImageNet datasets/imagenet

Train your own models

cd /path/to/your/SelfSup/examples/simclr/simclr.res50.scratch.imagenet.224size.256bs.200e
# pre-train
pods_train --num-gpus 8
# convert to weights
python convert.py simclr.res50.scratch.imagenet.224size.256bs.200e/log/model_final.pth weights.pkl
# downstream evaluation
cd /path/to/your/simclr.res50.scratch.imagenet.224size.256bs.200e.lin_cls
pods_train --num-gpus 8 MODEL.WEIGHTS /path/to/your/weights.pkl

Model Zoo

Supervised Classification

ImageNet

MethodsTraining ScheduleTop 1 Acc
Res50100e76.4

CIFAR 10

MethodsTraining ScheduleTop 1 Acc
Res50200e95.4

STL 10

MethodsTraining ScheduleTop 1 Acc
Res50150e86.1

Self-Supervised Learning - Classification

All results in the below table are trained using resnet-50 and reported on the ILSVRC2012 dataset.

MethodsTraining ScheduleBatch SizeOur Acc@1Official Acc@1
MoCo200e25660.560.5
MoCov2200e25667.667.5
SimCLR200e25663.261.9
SimCLR*200e25667.3Ours
SiMo200e25668.1Ours
SimSiam100e25667.667.7
SwAV200e25673.072.7
BYOL100e204869.866.5 (bs4096 from SimSiam paper)
BarlowTwins300e1024Comming Soon71.7

Self-Supervised Learning - Detection (2D)

All the results reported below are trained on ILSVRC2012 and evaluated on MS COCO using Faster-RCNN-FPN and resnet-50.

MethodsTraining ScheduleBatch SizeBox AP
SCRL200409639.9 ( official: 40.5 with bs 8192)
DetCon200256Comming Soon.

Self-Supervised Learning - 3D Scene Understanding

MethodsTraining ScheduleDownstream task
PointContrast-Comming Soon.

Citation

SelfSup is a part of cvpods, so if you find this repo useful in your research, or if you want to refer the implementations in this repo, please consider cite:


@article{zhu2020eqco,
  title={EqCo: Equivalent Rules for Self-supervised Contrastive Learning},
  author={Zhu, Benjin and Huang, Junqiang and Li, Zeming and Zhang, Xiangyu and Sun, Jian},
  journal={arXiv preprint arXiv:2010.01929},
  year={2020}
}

@misc{zhu2020cvpods,
  title={cvpods: All-in-one Toolbox for Computer Vision Research},
  author={Zhu*, Benjin and Wang*, Feng and Wang, Jianfeng and Yang, Siwei and Chen, Jianhu and Li, Zeming},
  year={2020}
}