DROID case study: How Stereolabs ZED cameras power groundbreaking robotics research at scale

This case study explores how Stereolabs' advanced perception systems helped overcome critical technical challenges in data collection, storage optimization, and depth estimation—enabling the creation of a dataset that's pushing the boundaries of real-world robot manipulation capabilities.

DROID in action

The challenge: building real-world robot capabilities through diverse data

Prior to DROID, even the most advanced robot manipulation systems struggled with generalization. The fundamental problem was data—or rather, the lack of sufficiently diverse, high-quality data collected outside controlled laboratory environments.

The Core Challenges:

1. Data Collection at Scale: Gathering robotic manipulation data across diverse environments presented significant logistical challenges
2. Hardware Standardization: Maintaining consistent hardware setups across 13 different institutions required reliable, easy-to-configure equipment
3. Data Volume Management: The team was collecting approximately 100GB of uncompressed multi-camera data per minute
4. Storage Optimization: Depth data is both massive and difficult to compress efficiently

"Before DROID, robot manipulation research was mostly stuck in simplified lab settings. We wanted to change that—to build a dataset that would let researchers take a robot, place it in a brand-new home, and see it do something useful" explains Alexander Khazatsky, CTO of CollectedAI.

After evaluating several vision systems, including Intel RealSense cameras, the team turned to Stereolabs for a solution.

The solution: standardized perception with Stereolabs ZED cameras

The DROID team implemented a standardized hardware configuration across all 13 participating institutions, featuring:

• A Franka Panda 7DoF robot arm
• Two adjustable Stereolabs ZED 2 stereo cameras (external scene perception)
• A wrist-mounted Stereolabs ZED Mini stereo camera (close manipulation view)
• An Oculus Quest 2 headset with controllers for teleoperation
• A portable, height-adjustable desk for rapid scene changes

The Stereolabs ZED SDK provided critical functionality that transformed how the team approached data collection and processing.

Key Implementation Advantages:

• Post-Processing Depth Generation: Rather than storing raw depth data, the ZED system allowed the team to generate depth information from compressed MP4 video after collection
• Optimized Data Pipeline: Built-in procedures for high-speed data writing solved bottlenecks without custom development
• Reliable Performance: Consistent operation across diverse environments with minimal calibration issues
• Cross-Platform SDK: Well-documented API that facilitated integration across the distributed research team

Results: transformative impact on robotics research

The integration of Stereolabs ZED cameras and SDK into the DROID platform yielded significant quantitative and qualitative benefits that transformed the project's scope and capabilities.

Quantitative Impacts:

• Data Storage Efficiency: The ability to generate depth from MP4 video dramatically reduced storage requirements while maintaining data quality
• Development Time Savings: "ZED already had highly optimized built-in procedures for exactly this, saving us weeks of work," notes Khazatsky
• Unprecedented Scale: Successfully collected 76,000 demonstration trajectories (350 hours) across 564 scenes and 86 tasks
• Hardware Reliability: Minimal downtime across 13 installation sites over 12 months of continuous data collection

Qualitative Impacts:

The integration of Stereolabs technology delivered substantial intangible benefits that enhanced the project's overall success:

"I choose ZED cameras because I want hardware I can rely on. That peace of mind has real value to me and my team. ZED delivers rugged, durable cameras—which matters during real-world deployment—paired with a stable, well-maintained SDK and excellent customer support. It's a rare combination that makes day-to-day work smoother and lets us move faster and more confidently."
— Alexander Khazatsky, CTO, CollectedAI

Unexpected Benefits:

Beyond the core challenges that Stereolabs helped solve, the team discovered additional advantages:

1. High-Quality IMU: "We didn't plan to use it since most of our cameras were fixed, but in new experiments involving mobile systems, it's been a great built-in asset," shares Khazatsky
2. Versatile Environmental Operation: "Unlike IR-based methods, ZED's stereo cameras can even operate in settings like aquariums. I've seen some cool research happening in that domain."

Future-Proof Dataset: The ability to reprocess data with improved stereo-to-depth models as technology advances maintains the dataset's long-term value

Why Stereolabs outperformed the competition

The DROID team evaluated several alternatives before settling on Stereolabs as their perception partner of choice:

"We tried various Intel RealSense models and considered other stereo alternatives. ZED consistently outperformed in reliability, API quality, and support."
— Alexander Khazatsky, CTO, CollectedAI

Key differentiators included:

1. Superior Depth Estimation Approach: ZED's stereo vision approach provided more flexibility than IR-based alternatives
2. Storage Optimization: "Storage space was a critical challenge. Depth data is massive and hard to compress, but ZED's ability to synchronize dual stereo color streams using their SVO format to generate depth after capture gave us a highly compressed and efficient storage method without significant loss in quality."
3. Well-Documented SDK: "The ZED SDK is powerful, well-documented, and easy to use. Any time I wanted to experiment, I found the information I needed within minutes."
4. Responsive Support: "On the rare occasions I needed help, their support team responded rapidly. All of this saved our team significant time during both development and deployment."

Looking forward: the future of robotics and 3D vision

The success of the DROID project demonstrates the critical importance of high-quality perception systems in advancing robotic capabilities. For projects pushing the boundaries of robotic manipulation, the quality of visual data remains a fundamental requirement.

"Robotics is fundamentally data-hungry, and long-term success depends on high-quality, forward-compatible data. Innovation in 3D vision is constant, so we prioritize methods that give us flexibility to adapt to future progress. That's why we care so much about storing data in ways that remain useful years down the line." — Alexander Khazatsky, CTO, CollectedAI

The collaboration between the DROID initiative and Stereolabs exemplifies how cutting-edge 3D vision technology can solve complex perception challenges while creating new possibilities in robotics research.

Looking ahead, the partnership continues: "At CollectedAI, we're developing large-scale, experimental data collection systems—and ZED cameras will continue to be a trusted component in that pipeline."

Conclusion

The DROID initiative's success hinged on overcoming significant technical challenges in data collection and management. By implementing Stereolabs ZED cameras and leveraging their advanced SDK, the team created one of the most comprehensive robot manipulation datasets ever assembled—one that's actively advancing the field's capabilities in real-world settings.

For robotics researchers and companies facing similar challenges with large-scale data collection, depth estimation, or storage optimization, Stereolabs' solutions offer proven capabilities that outperform alternatives while providing the reliability and support necessary for mission-critical applications.

Click here for more information about the DROID dataset.