Probabilistic_semantic_mapping

Stacks include:

• aizo_quadrotor_slam: 2D SLAM based on CaroGrapher and TEB motion planning
• oj_detection: tracker and extracted object, real-time object detection using the Ultralytics YOLO
• The object_detector_node provides real-time object detection on incoming ROS image messages using the Ultralytics YOLO model.
• The tracker_with_cloud_node provides functionality for 3D object detection by integrating 2D detections, LiDAR data, and camera information. Check each package for more details.

Our proposed:

input image	point cloud	tracker object	cluster point	traditional map	semantic map

Citation:

Object-Oriented Semantic Mapping for Reliable UAVs Navigation, ICCAIS 2023

Setup ⚙

$ cd ~/catkin_ws/src
$ git clone https://github.com/NguyenCanhThanh/probabilistic_semantic_mapping.git
$ python3 -m pip install -r oj_detection/requirements.txt
$ cd ~/catkin_ws
$ rosdep install -r -y -i --from-paths .
$ catkin build

Requirements

Requirements

• ros (indigo+)
• gazebo (2.2+)
• gazebo_ros (2.2+)
• quadrotor_control KumarRobotics
• qudrotor_msgs KumarRobotics
• waypoint_navigation .
• aizo_quadrotor NCT

Step 1: Custom dataset

Step 2: SLAm node

- CartoGrapher build static map
roslaunch aizo_quadrotor_slam 2dslam.launch

- Move based navigation using TEB

roslaunch aizo_quadrotor_slam move_base.launch 

tracker_node

Params

• yolo_model: Pre-trained Weights.
  For yolov8, you can choose yolov8*.pt, yolov8*-seg.pt, yolov8*-pose.pt.

  YOLOv8
  YOLOv8-seg
  YOLOv8-pose

  See also: https://docs.ultralytics.com/models/

• image_topic: Topic name for image.

• detection_topic: Topic name for 2D bounding box.

• conf_thres: Confidence threshold below which boxes will be filtered out.

• iou_thres: IoU threshold below which boxes will be filtered out during NMS.

• max_det: Maximum number of boxes to keep after NMS.

• tracker: Tracking algorithms.

• classes: List of class indices to consider.
  See also: https://github.com/ultralytics/ultralytics/blob/main/ultralytics/datasets/coco128.yaml

• debug: If true, run simple viewer.

• debug_conf: Whether to plot the detection confidence score.

• debug_line_width: Line width of the bounding boxes.

• debug_font_size: Font size of the text.

• debug_labels: Font to use for the text.

• debug_font: Whether to plot the label of bounding boxes.

• debug_boxes: Whether to plot the bounding boxes.

Topics

• Subscribed Topics:
  • Image data from image_topic parameter. (sensor_msgs/Image)
• Published Topics:
  • Debug images to /debug_image topic. (sensor_msgs/Image)
  • Detected objects(2D bounding box) to detection_topic parameter. (vision_msgs/Detection2DArray)

tracker_with_cloud_node

Params

• camera_info_topic: Topic name for camera info.
• point_topic: Topic name for point cloud.
• detection2d_topic: Topic name for 2D bounding box.
• detection3d_topic: Topic name for 3D bounding box.
• cluster_tolerance: Spatial cluster tolerance as a measure in the L2 Euclidean space.
• min_cluster_size: Minimum number of points that a cluster needs to contain.
• max_cluster_size: Maximum number of points that a cluster needs to contain.

Topics

• Subscribed Topics:
  • Camera info from camera_info_topic parameter. (sensor_msgs/CameraInfo)
  • Lidar data from point_topic parameter. (sensor_msgs/PointCloud2)
  • Detected objects(2D bounding box) from detection2d_topic parameter. (vision_msgs/Detection2DArray)
• Published Topics:
  • Detected cloud points to /detection_cloud topic. (sensor_msgs/PointCloud2)
  • Projected cloud points to /projection_cloud topic. (sensor_msgs/PointCloud2)
  • Detected objects(3D bounding box) to detection3d_topic parameter. (vision_msgs/Detection3DArray)
  • Visualization markers to /detection_marker topic. (visualization_msgs/MarkerArray)