PASSL reproduces MoCo, a way of building large and consistent dictionaries for self-supervised learning with a contrastive loss. It also includes the higher baseline MoCo v2

• See INSTALL.md

• See GETTING_STARTED.md

Models are all trained with ResNet-50 backbone.

python tools/train.py -c configs/moco/moco_v[1,2]_r50.yaml

python -m paddle.distributed.launch --gpus="0,1,2,3,4,5,6,7" tools/train.py -c configs/moco/moco_v[1,2]_r50.yaml

Pretraining models with 200 epochs can be found at MoCo v1 and MoCo v2

Note: The default learning rate in config files is for 8 GPUs. If using differnt number GPUs, the total batch size will change in proportion, you have to scale the learning rate following new_lr = old_lr * new_ngpus / old_ngpus. Replacing v[1,2] to v1 or v2 according to your requriement.

python tools/extract_weight.py ${CHECKPOINT} --output ${WEIGHT_FILE}

• Support PaddleClas

Convert the format of the extracted weights to the corresponding format of paddleclas to facilitate training on paddleclas

python tools/passl2ppclas/convert.py --type res50 --checkpoint ${CHECKPOINT} --output ${WEIGHT_FILE}

Note: It must be ensured that the weights are extracted

python -m paddle.distributed.launch --gpus="0,1,2,3,4,5,6,7" tools/train.py -c configs/moco/moco_clas_r50.yaml --pretrained ${WEIGHT_FILE}

python -m paddle.distributed.launch --gpus="0,1,2,3,4,5,6,7" tools/train.py -c configs/moco/moco_clas_r50.yaml --load ${CLS_WEGHT_FILE} --evaluate-only

The trained linear weights in conjuction with the backbone weights can be found at MoCo v1 linear and MoCo v2 linear