A ROS package to use LibTorch, a PyTorch C API, for inference with our scene recognition models.

A Docker environment for running this package is here. This package is only tested in the virtual environment.

• PyTorch with LibTorch built
  • Tested with 1.5.0
• semantic_segmentation_srvs/GetLabelAndProbability

A node to use a multi-task model for semantic segmentation, traversability estimation, and path estimation.

• image (sensor_msgs/Image)

  An input image

• label (sensor_msgs/Image)

  Image that stores label indices of each pixel

• color_label (sensor_msgs/Image)

  Image that stores color labels of each pixel (for visualization)

• prob (sensor_msgs/Image)

  Image that stores traversability of each pixel

• start_point (geometry_msgs/PointStamped)

  Start point of the estimated path line

• end_point (geometry_msgs/PointStamped)

  End point of the estimated path line

• get_label_image (semantic_segmentation_srvs/GetLabelAndProbability)

  Return inference results (segmentation and traversability) for a given image.

• image (sensor_msgs/Image)

  An input image

• start_point (geometry_msgs/PointStamped)

  Start point of the estimated path line from the inference node

• end_point (geometry_msgs/PointStamped)

  End point of the estimated path line from the inference node

• image_with_path (sensor_msgs/Image)

  An image with the path overlaid

roslaunch pytorch_enet_ros.launch image:=<image topic name> model_name:=<model name>

The ROS nodes in this package use models saved as a serialized Torch Script file.

At this moment, we don't provide a script to generate the weight files.

Refer to this page to get the weight file.

If the version of PyTorch that runs this ROS package and that you generate your weight file (serialized Torch Script) do not match, the ROS node may fail to import the weights.

For example, if you use our Docker environment, the weights should be generated using PyTorch 1.5.0.

For visualization of semantic segmentation, we use a color map image.

It is a 1xC PNG image file (C: The number of classes), where the color of class i is stored in the pixel at (1, i).

This repository is used in experiments in the publication as follows:

Shigemichi Matsuzaki, Hiroaki Masuzawa, Jun Miura, Image-Based Scene Recognition for Robot Navigation Considering Traversable Plants and Its Manual Annotation-Free Training, IEEE Access, vol. 10, pp. 5115-5128, 2022 [paper]