Pedestrian Detection and Tracking
HumaNet is a program that automatically detects and disambiguates humans in a video. HumaNet uses Faster Region-based Convolutional Neural Networks (Faster R-CNN) to detect human entities in a video frame, then trace their movement using Simple Online and Realtime Tracking with a Deep Association Metric (Deep SORT) that learn unique characteristics of each individual movement. Savitzky–Golay Filter is used to smooth the trajectories for visualization.
Honorable Mention in BEST2018 Human Detection (Open Competition) from "Humanet", National Software Contest (NSC) 2018
Poom Wettayakorn and Asst. Prof. Dr. Suppawong Tuarob (Advisor), I-IUSKi Lab
Faster R-CNN: Faster Region-based Convolutional Nueral Networks
Please download the object detection model file here, and place it to ./object_detection/faster_rcnn_resnet101_coco_11_06_2017/
Deep SORT: Simple Online and Realtime Tracking with a Deep Association Metric
Kalman Filter & Hungarian Algorithm
Linear Interpolation and Savitzky-Golay Filter
HumaNet
object_detection
faster_rcnn_resnet101_coco_11_06_2017
-realtime_detection.py
-Object_Detection_Demo.ipynb
object_tracking
deep_sort
-1_detection.py
-2_generate_features.py
-3_deep_sort.py
-4_concatenate.py
-5_show_results.py
-pipeline.sh
-scatter_plot.ipynb
dataset
captures
BEST_1
detections
features
outputs
-BEST_1_opt.mp4
-BEST_1_reg.mp4
tracks
videos
-BEST_1.mp4
-20p32w0047.txt
Realtime_detection.py
- Detection demo from video fileObject_Detection_Demo.ipynb
- Detection demo from image file (Running on Jupyter notebook)Pipeline.sh
- Pipeline shell script that creates a set of python scripts to run the demo from beginning to end.Scatter_plot.ipynb
- 3D Trajectory visualization (Running on Jupyter notebook)
- Python 3.5
- pip3 - tool for installing Python packages.
- GPU card with CUDA Compute Capability 3.0 or higher NVIDIA's documentation
- Install CUDA Toolkit 8.0
- Download cuDNN v5.1
- Install TensorFlow with GPU (Note: Our current detection model is compatible for TensorFlow version 1.1)
- How to use CUDA and the GPU Version of Tensorflow
- Install OpenCV from Source
- Ubuntu 16.04: How to install OpenCV
- Install OpenCV3 on Ubuntu
- Install Python Packages
cd HumaNet/
for req in $(cat requirements.txt); do sudo pip3 install $req; done
- Plotly - Registration and Initialization for Online Plotting
- Detection Demo from video
cd object_detection
python3 realtime_detection.py
- Detection Demo from image - running on Jupyter notebook
cd object_detection
jupyter notebook
- Click Object_Detection_Demo.ipynb file from jupyter’s interface
- Running Whole System via Shell Script
cd object_tracking
./pipeline.sh
NOTE: After running pipeline.sh output files will be stored at dataset/outputs (Video file) and dataset/captures (Image file)
-
Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’14). IEEE Computer Society, Washington, DC: 580-587. DOI:http://dx.doi.org/10.1109/CVPR.2014.81
-
Ross Girshick. 2015. Fast R-CNN. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV '15). IEEE Computer Society, Washington, DC: 1440-1448. DOI:http://dx.doi.org/10.1109/ICCV.2015.169
-
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett (Eds.), Advances in Neural Information Processing Systems 28-Proceedings of the Neural Information Processing Systems Conference (NIPS 2015). Neural Information Processing Systems Foundation, Inc. http://papers.nips.cc/paper/5638-faster-r-cnn-towards-real-time-object-detection-with-region-proposal-networks
-
Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, and Kevin Murphy. 2017. Speed/accuracy trade-offs for modern convolutional object detectors. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR 2017). The Computer Vision Foundation: 7310-7319. http://openaccess.thecvf.com/CVPR2017.py
-
J. R. R. Uijlings, K. E. A. van de Sande, T. Gevers, and A. W. M. Smeulders. 2013. Selective Search for Object Recognition. International Journal of Computer Vision 104, 2: 154-171. DOI:http://dx.doi.org/10.1007/s11263-013-0620-5
-
A. Bewley, G. Zongyuan, F. Ramos, and B. Upcroft. “Simple Online and Realtime Tracking,” in ICIP, 2016, pp. 3464–3468.
-
L. Leal-Taixé, A. Milan, I. Reid, S. Roth, and K. Schindler, “MOTChallenge 2015: Towards a Benchmark for Multi-target Tracking,” arXiv:1504.01942 [cs], 2015.
-
A. Milan, L. Leal-Taixé, I. Reid, S. Roth, and K. Schindler, “Mot16: A Benchmark for Multi-object Tracking,” arXiv preprint arXiv:1603.00831, 2016.
-
F. Yu, W. Li, Q. Li, Y. Liu, X. Shi, J. Yan. POI: Multiple Object Tracking with High Performance Detection and Appearance Feature. In BMTT, SenseTime Group Limited, 2016.
-
N. Wojke, A. Bewley, and D. Paulus. “Simple Online and Realtime Tracking with a Deep Association Metric,” IEEE International Conference on Image Processing (ICIP), 2017, pp. 3645–3659.
-
Aniket Bera, Sujeong Kim, Tanmay Randhavane, Srihari Pratapa, and Dinesh Manocha. “GLMP- realtime pedestrian path prediction using global and local movement patterns,” ICRA, 2016.