Self-Supervised Learning for Multimodal Robot Perception with OctoSense 🔥🔥 One platform, eight sensors, one clock, synchronized multimodal driving across day, night, and degraded… | Nicolai Nielsen | 35 comments
Self-Supervised Learning for Multimodal Robot Perception with OctoSense 🔥🔥 One platform, eight sensors, one clock, synchronized multimodal driving across day, night, and degraded conditions. Self-supervised foundation models, DINO, SigLIP, V-JEPA, transformed robot perception, but they are vision-only, and in the real world no single sensor suffices. Cameras degrade under low light, high dynamic range, and rapid motion; LiDAR is accurate but sparse with poor semantics. Every sensor has different rates, resolutions, noise, and failure modes, and they fail in different ways. Robust robot perception needs representations that survive these failures, yet self-supervised learning has stayed almost entirely on vision and text. OctoSense aligns all the sensors to a single timeline using our PPS time-sync hardware, a unique six-pulse identifier every four minutes and fifteen seconds lets every stream realign even after a dropped trigger. At native rates the platform produces ~1.7 GB/s; on-board compression (LiDAR/event packets, H.265 video) cuts that 21× to 78.7 MB/s with no dropped data. Calibration uses a retro-reflective circle on an AprilGrid jointly visible to the cameras and LiDAR. 👉 Check it out here: https://lnkd.in/evSCASpQ | 35 comments on LinkedIn
https://www.linkedin.com/posts/nicolaiai_self-supervised-learning-for-multimodal-robot-activity-7479135098022379520-hCvY/
Seonglae Cho
