Comparisons of top-1 accuracies on the validation set of ImageNet, by using ResNet50 trained with SN, BN, and GN in different batch size settings. The bracket (·, ·) denotes (#GPUs,#samples per GPU). In the bottom part, “GN-BN” indicates the difference between the accuracies of GN and BN. The “-” in (8, 1) of BN indicates it does not converge.
| (8,32) | (8,16) | (8,8) | (8,4) | (8,2) | (1,16) | (1,32) | (8,1) | (1,8) | |
| BN | 76.4 | 76.3 | 75.2 | 72.7 | 65.3 | 76.2 | 76.5 | – | 75.4 |
| GN | 75.9 | 75.8 | 76.0 | 75.8 | 75.9 | 75.9 | 75.8 | 75.5 | 75.5 |
| SN | 76.9 | 76.7 | 76.7 | 75.9 | 75.6 | 76.3 | 76.6 | 75.0* | 75.9 |
| GN−BN | -0.5 | -0.5 | 0.8 | 3.1 | 10.6 | -0.3 | -0.7 | – | 0.1 |
| SN−BN | 0.5 | 0.4 | 1.5 | 3.2 | 10.3 | 0.1 | 0.1 | – | 0.5 |
| SN−GN | 1.0 | 0.9 | 0.7 | 0.1 | -0.3 | 0.4 | 0.8 | -0.5 | 0.4 |
- Install PyTorch
- Clone the repo:
git clone https://github.com/switchablenorms/Switchable-Normalization.git
- python packages
- pytorch>=0.4.0
- torchvision>=0.2.1
- tensorboardX
- pyyaml
- Download the ImageNet dataset and put them into the
{repo_root}/imagenet/data/imagenet. The meta file(train.txt or val.txt) consists of the file name and category. Just like:n01440764/n01440764_10026.JPEG 0 n01440764/n01440764_10027.JPEG 0 n01440764/n01440764_10029.JPEG 0 n01440764/n01440764_10040.JPEG 0 n01440764/n01440764_10042.JPEG 0 n01440764/n01440764_10043.JPEG 0
./train_val.sh configs/config_resnetv1sn50_cosine.yaml
Download the pretrained models from Model Zoo and put them into the {repo_root}/data/pretrained_model
./test.sh configs/config_resnetv1sn50_cosine.yaml
We provide models pretrained with SN on ImageNet, and compare to those pretrained with BN as reference. If you use these models in research, please cite the SN paper. The configuration of SN is denoted as (#GPUs, #images per GPU).
| Model | Top-1* | Top-5* | Epochs | LR Scheduler | Weight Decay | Download |
|---|---|---|---|---|---|---|
| ResNet101v2+SN (8,32) | 78.81% | 94.16% | 120 | warmup + cosine lr | 1e-4 | [Google Drive] [Baidu Pan] |
| ResNet101v1+SN (8,32) | 78.54% | 94.10% | 120 | warmup + cosine lr | 1e-4 | [Google Drive] [Baidu Pan] |
| ResNet50v2+SN (8,32) | 77.57% | 93.65% | 120 | warmup + cosine lr | 1e-4 | [Google Drive] [Baidu Pan] |
| ResNet50v1+SN (8,32) | 77.49% | 93.32% | 120 | warmup + cosine lr | 1e-4 | [Google Drive] [Baidu Pan] |
| ResNet50v1+SN (8,32) | 76.92% | 93.26% | 100 | Initial lr=0.1 decay=0.1 steps[30,60,90,10] | 1e-4 | [Google Drive] [Baidu Pan] |
| ResNet50v1+SN (8,4) | 75.85% | 92.7% | 100 | Initial lr=0.0125 decay=0.1 steps[30,60,90,10] | 1e-4 | [Google Drive] [Baidu Pan] |
| ResNet50v1+SN (8,1)† | 75.94% | 92.7% | 100 | Initial lr=0.003125 decay=0.1 steps[30,60,90,10] | 1e-4 | [Google Drive] [Baidu Pan] |
| ResNet50v1+BN | 75.20% | 92.20% | -- | stepwise decay | -- | [TensorFlow models] |
| ResNet50v1+BN | 76.00% | 92.98% | -- | stepwise decay | -- | [PyTorch Vision] |
| ResNet50v1+BN | 75.30% | 92.20% | -- | stepwise decay | -- | [MSRA] |
| ResNet50v1+BN | 75.99% | 92.98% | -- | stepwise decay | -- | [FB Torch] |
*single-crop validation accuracy on ImageNet (a 224x224 center crop from resized image with shorter side=256)
†For (8,1), SN contains IN and LN without BN, as BN is the same as IN in training. When using this model, you should add using_bn : False in yaml file.
In evaluation, download the above models and put them into the {repo_root}/imagenet/data/pretrained_model.