Each Indoor Sensor Node(ISN) has two Cameras and a LiDAR.

Overall Goal

• [x] Object Detection and Tracking for all objects in a Hospital Hallway.
• [ ] Re-ID and fusion of Tracks for multiple sensor nodes in the cloud.
• [ ] Planning

Pipeline(Aryan + Shashwat)

                                                                      **ISN Perception Pipeline**

LiDAR Perception(Aryan)

                                                                    **LiDAR Perception Pipeline**

FILTER

• ROI Filtering
• Voxel Downsampling

CLUSTER

• DBSCAN

TRACK

• Kalman Filter
• Hungarian Algorithm

Camera Perception(Aryan)

                                                                **Camera Perception Pipeline**

OBJ DETECTION & TRACKING

• YOLO v8 POSE (for now)
• BoT SORT

GROUND CONTACT POINTS

• Feet, Wheel, etc Detection

3D PROJECTION

• Project from Image Plane to LiDAR Plane

FUSION(Aryan + Minghao)

                                                                            **Fusion Pipeline**

RECONSTRUCTION