CVPR | paperswithcode

Pytorch implementation of the SketchGAN paper.
SketchGAN: Joint Sketch Completion and Recognition with Generative Adversarial Network
Fang Liu^1,2, Xiaoming Deng¹, Yu-Kun Lai³, Yong-Jin Liu^{4 *}, Cuixia Ma^{1,2 *}, Hongan Wang^{1,2 *},
¹State Key Laboratory of Computer Science and Beijing Key Lab of Human-Computer Interaction, Institute of Software, Chinese Academy of Sciences, ²University of Chinese Academy of Sciences, ³Cardiff University, ⁴Tsinghua University
In CVPR 2019.

SketchGAN is a new generative adversarial network (GAN) based approach that jointly completes and recognizes a sketch, boosting the performance of both tasks. It is used for hand-drawn sketch recognition which is a fundamental problem in computer vision, widely used in sketch-based image and video retrieval, editing, and reorganization¹.

The paper proposes a cascade Encode-Decoder network to complete the input sketch in an iterative manner, and employs an auxiliary sketch recognition task to boost the performance of both tasks¹.

• Python 3
• CPU or NVIDIA GPU CUDA CuDNN

• Clone this repo:

  git clone [email protected]:Ashish-Abraham/Sketch_GAN.git
  cd Sketch_GAN

• Install PyTorch and other dependencies (e.g., torchvision).

  pip install -r requirements.txt

• Training image directory should be of the following structure

├── image_paths.csv

├── trainA
  ....
├── trainB
  ....
├── valA
  ....
└── valB
  ....

• Folders with A suffix contain corrupted sketches and those with B contain target sketches.
• The GAN should map A->B.
• Each image.png in __B folder contains corresponding imagec.png in __A folder.
• Use data_csv_script.py to generate corresponding csv file from dataset for training.

The repo contains all code required to implement the GAN. Edit the scripts given or export the code in scripts to a jupyter notebook appropriately to train. Make sure cuda is available.

Input	Ground Truth	Output

F. Liu, X. Deng, Y. -K. Lai, Y. -J. Liu, C. Ma and H. Wang, "SketchGAN: Joint Sketch Completion and Recognition With Generative Adversarial Network," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 5823-5832, doi: 10.1109/CVPR.2019.00598.

Yu, Q., Yang, Y., Liu, F. et al. Sketch-a-Net: A Deep Neural Network that Beats Humans. Int J Comput Vis 122, 411–425 (2017). https://doi.org/10.1007/s11263-016-0932-3