国内资源链接(密码：e2ww)

• SRD
• ISTD
• ISTD DA, SRD DA

• Results on SRD: TBRNet_SRD
• Results on SRD DA: TBRNet_SRD_DA
• Results on ISTD: TBRNet_ISTD
• Results on ISTD DA: TBRNet_ISTD_DA
• Results on ISTD : TBRNet_ISTD

Visual comparison results of penumbra removal results on the SRD dataset - (Powered by MulimgViewer)

• Model on SRD: TBRNet_SRD.pth
• Model on SRD DA: TBRNet_SRD_DA.pth
• Model on ISTD: TBRNet_ISTD.pth
• Model on ISTD DA: TBRNet_ISTD_DA.pth
• Model on ISTD : TBRNet_ISTD .pth

1. MAE (i.e., RMSE in paper). Hieu Le reuploaded the evaluation code. Before that, everyone can obtain this code from Jifeng Wang via email. Currently, the MATLAB code is used in most state-of-the-art works for shadow removal.

2. PSNR SSIM. zhu_aaai2022

ubuntu18.04 cuda10.2 pytorch1.7.1

1. create environments

conda env create -f install.yaml

2. activate environments

conda activate TBRNet

For example, generate the dataset list of ISTD:

1. Download:
   • ISTD and SRD
   • USR shadowfree images
   • Syn. Shadow
   • SRD shadow mask (or try our re-uploaded link)
   • train_B_ISTD:

   cp -r ISTD_Dataset_arg/train_B ISTD_Dataset_arg/train_B_ISTD
   cp -r ISTD_Dataset_arg/train_B SRD_Dataset_arg/train_B_ISTD
   

   • VGG19

   cp vgg19-dcbb9e9d.pth ISTD_Dataset_arg/
   cp vgg19-dcbb9e9d.pth SRD_Dataset_arg/
   

2. The data folders should be:

   ISTD_Dataset_arg
       * train
           - train_A # ISTD shadow image
           - train_B # ISTD shadow mask
           - train_C # ISTD shadowfree image
           - shadow_free # USR shadowfree images
           - synC # Syn. shadow
           - train_B_ISTD # ISTD shadow mask
       * test
           - test_A # ISTD shadow image
           - test_B # ISTD shadow mask
           - test_C # ISTD shadowfree image
       * vgg19-dcbb9e9d.pth
   
   SRD_Dataset_arg
       * train #  renaming the original `Train` folder in `SRD`.
           - train_A # SRD shadow image, renaming the original `shadow` folder in `SRD`.
           - train_B # SRD shadow mask
           - train_C # SRD shadowfree image, renaming the original `shadow_free` folder in `SRD`.
           - shadow_free # USR shadowfree images
           - synC # Syn. shadow
           - train_B_ISTD # ISTD shadow mask
       * test #  renaming the original `test_data` folder in `SRD`.
           - train_A # SRD shadow image, renaming the original `shadow` folder in `SRD`.
           - train_B # SRD shadow mask
           - train_C # SRD shadowfree image, renaming the original `shadow_free` folder in `SRD`.
       * vgg19-dcbb9e9d.pth 
   

3. Edit generate_flist_istd.py: (Replace path)

ISTD_path = "/Your_data_storage_path/ISTD_Dataset_arg"

4. Generate Datasets List. (Already contains ISTD DA.)

conda activate TBRNet
cd script/
python generate_flist_istd.py

5. Edit config_ISTD.yml: (Replace path)

DATA_ROOT: /Your_data_storage_path/ISTD_Dataset_arg

For example, training test evaluation on ISTD dataset.

cp config/config_ISTD.yml config.yml 
cp config/run_ISTD.py run.py
conda activate TBRNet
python run.py

For example, test evaluation on ISTD dataset.

1. Download weight file(TBRNet_ISTD.pth) to pre_train_model/ISTD
2. Copy file

cp config/config_ISTD.yml config.yml 
cp config/run_ISTD.py run.py
mkdir -p checkpoints/ISTD/
cp config.yml checkpoints/ISTD/config.yml
cp pre_train_model/ISTD/TBRNet_ISTD.pth  checkpoints/ISTD/ShadowRemoval.pth

3. Edit run.py. Comment the training code.

    # # pre_train (no data augmentation)
    # MODE = 0
    # print('\nmode-' str(MODE) ': start pre_training(data augmentation)...\n')
    # for i in range(1):
    #     skip_train = init_config(checkpoints_path, MODE=MODE,
    #                             EVAL_INTERVAL_EPOCH=1, EPOCH=[90,i])
    #     if not skip_train:
    #         main(MODE, config_path)
    # src_path = Path('./pre_train_model') / \
    #     config["SUBJECT_WORD"]/(config["MODEL_NAME"] '_pre_da.pth')
    # copypth(dest_path, src_path)

    # # train
    # MODE = 2
    # print('\nmode-' str(MODE) ': start training...\n')
    # for i in range(1):
    #     skip_train = init_config(checkpoints_path, MODE=MODE,
    #                             EVAL_INTERVAL_EPOCH=0.1, EPOCH=[60,i])
    #     if not skip_train:
    #         main(MODE, config_path)
    # src_path = Path('./pre_train_model') / \
    #     config["SUBJECT_WORD"]/(config["MODEL_NAME"] '_final.pth')
    # copypth(dest_path, src_path)

4. Run

conda activate TBRNet
python run.py

After evaluation, execute the following code to display the final RMSE.

python show_eval_result.py

Output:

running rmse-shadow: xxx, rmse-non-shadow: xxx, rmse-all: xxx # ISRD

This is the evaluation result of python pytorch, which is only used during training. To get the evaluation results in the paper, you need to run the matlab code.

Part of the code is based upon:

• https://github.com/nachifur/LLPC
• https://github.com/vinthony/ghost-free-shadow-removal
• https://github.com/knazeri/edge-connect

If you find our work useful in your research, please consider citing:

@ARTICLE{liu2023shadow,
  author={Liu, Jiawei and Wang, Qiang and Fan, Huijie and Tian, Jiandong and Tang, Yandong},
  journal={IEEE Transactions on Neural Networks and Learning Systems}, 
  title={A Shadow Imaging Bilinear Model and Three-Branch Residual Network for Shadow Removal}, 
  year={2023},
  volume={},
  number={},
  pages={1-15},
  doi={10.1109/TNNLS.2023.3290078}
}

Please contact Jiawei Liu if there is any question ([email protected]).