Embodied-sync: Synchronize, calibrate, align, and validate multimodal robot-learning data across sensors, clocks, live sessions, and recordings

<aside class="onebox githubrepo"> <header class="source"> <a href="https://github.com/anicut-ai/embodied-sync" rel="noopener nofollow ugc" target="_blank">github.com</a> </header> <article class="onebox-body"> <div class="github-row"> <img class="thumbnail" height="344" src="https://us1.discourse-cdn.com/flex022/uploads/ros/optimized/3X/e/1/e18bc3084f1271d1e679a4fca0f7ccb67583feb6_2_690x344.png" width="690" /> <h3><a href="https://github.com/anicut-ai/embodied-sync" rel="noopener nofollow ugc" target="_blank">GitHub - anicut-ai/embodied-sync: Synchronize, calibrate, align, and validate...</a></h3> <p><span class="github-repo-description">Synchronize, calibrate, align, and validate multimodal robot-learning data across sensors, clock domains, live sessions, and recorded datasets.</span></p> </div> </article> <div class="onebox-metadata"> </div> <div style="clear: both;"></div> </aside> <p>Often, robot-learning sensor streams or datasets need to be synchronized. Cameras run at one rate, robot state at another, packets arrive late, and a device reconnect can silently reset its time offset. When you are trying to finish an experiment, the last thing you need is to discover after training that the observations were paired differently on the robot than they were in the dataset.</p> <p><code>embodied-sync</code> gives you one place to align, replay, inspect, and validate multimodal timing. It works with both live sensor streams and recordings, and it fits around the tools you already use: UMI, LeRobot, ROS 2/rosbag2 + MCAP, LSL/XDF, Rerun, and SurgSync-style datasets. It also attempts to adapt to any custom dataset format, with an interactive dialog allowing users to pick the right choices for alignment/format. It also generates reports with detailed information like downsampled frequencies, dropped frames and latency and provides the ability to inspect frames/audio to verify.</p> <p></p><div class="lightbox-wrapper"><a class="lightbox" href="https://us1.discourse-cdn.com/flex022/uploads/ros/original/3X/a/3/a3030917d24c63755fad4827b404577142cbea23.jpeg" rel="noopener nofollow ugc" title="image"><img alt="image" height="350" src="https://us1.discourse-cdn.com/flex022/uploads/ros/optimized/3X/a/3/a3030917d24c63755fad4827b404577142cbea23_2_690x350.jpeg" width="690" /></a></div><p></p> <p>Current alpha users include QUT Robotics, hobby enthusiasts working with camera/IMU streams/tactile device streams, robotics startups. The methods have been validated on several real-world datasets including RocSync, SurgSync, QUT manipulation learning datasets and ROSBag2.</p> <p><small>1 post - 1 participant</small></p> <p><a href="https://discourse.openrobotics.org/t/embodied-sync-synchronize-calibrate-align-and-validate-multimodal-robot-learning-data-across-sensors-clocks-live-sessions-and-recordings/57766">Read full topic</a></p>