In this project, we use BiSeNet as our backbone network to finish scene segmentation task, then use Connected Components Decision to Postprocess scene segmentaion results to finish instance segmentation task.
The details can be accessed in CV_Poster.pdf.
Bilateral Segmentation Network (BiSeNet) is designed to segment scenes in Real-time, proposed by Face .
- PAPER: BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation
- CODE: https://github.com/ooooverflow/BiSeNet
The BiSeNet architure is shown as below.
To extend scene segmentation to instance segmentation, we use 8-connected components decision to postprocess the BiSeNet outputs. The connected components theory is shown as below.
The idea is quite easy.
- As the segmentation result of each class is a Binary Image, Connected Components Decision can be used to segment the class superpixel to instance superpixels.
- Some noises can be suppressed by removing some very small connected components.
Database links:
Models can be found at Baiduyun.
| Raw Image | Ground Truth | Scene | Instance |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
| Raw Image | Ground Truth | Scene | Instance |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Bisenet: Bilateral segmentation network for real-time semantic segmentation. Yu, Changqian, et al. European Conference on Computer Vision. Springer, Cham, 2018. (https://arxiv.org/pdf/1808.00897.pdf)
Sun rgb-d: A rgb-d scene understanding benchmark suite. Song, Shuran, Samuel P. Lichtenberg, and Jianxiong Xiao. Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. (http://rgbd.cs.princeton.edu/paper.pdf)
Scene Parsing through ADE20K Dataset. Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso and Antonio Torralba. Computer Vision and Pattern Recognition (CVPR), 2017. (http://people.csail.mit.edu/bzhou/publication/scene-parse-camera-ready.pdf)
@inproceedings{zhou2017scene,
title={Scene Parsing through ADE20K Dataset},
author={Zhou, Bolei and Zhao, Hang and Puig, Xavier and Fidler, Sanja and Barriuso, Adela and Torralba, Antonio},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year={2017}
}