Top row (left to right): Nancy M. Amato, Seth Hutchinson, and Ken Goldberg. Bottom row (left to right): Animesh Garg, Aude Billard, Russ Tedrake, and Frank Park. | Source: Science Robotics
Since its inception, the robotics industry has worked towards creating machines that could handle complex tasks by combining mathematical models with advanced computation. Now, the community finds itself divided on how to best reach that goal.
A group of roboticists from around the world investigated this divide at the IEEE International Conference on Robotics and Automation (ICRA) earlier this year. The show closed with a debate between six leading roboticists:
- Daniela Rus, who is the CSAIL director and the Andrew (1956) and Erna Viterbi Professor of Electrical Engineering and Computer Science. Rus also keynoted the Robotics Summit & Expo earlier this year.
- Russ Tedrake, who is the Toyota Professor at CSAIL, EECS, and the Department of Aeronautics and Astronautics.
- Leslie Kaelbling, who is the Panasonic Professor of Computer Science and Engineering at MIT.
- Aude Billard, a professor at the School of Engineering at the Swiss Federal Institute of Technology in Lausanne (EPFL).
- Frank Park, a professor of Mechanical Engineering at Seoul National University.
- Animesh Garg, a Stephen Fleming Early Career Assistant Professor at the School of Interactive Computing at Georgia Tech.
UC Berkeley’s Ken Goldberg moderated the debate, framing the discussion with the question: “Will the future of robotics be written in code or in data?”
The argument for a data-first approach
Daniela Rus giving a keynote talk at the Robotics Summit & Expo.
Rus and Tedrake argued that data-driven approaches, particularly those powered by large-scale machine learning, are critical to unlocking robots’ ability to function reliably in the real world.
“Physics gives us clean models for controlled environments, but the moment we step outside, those assumptions collapse,” Rus said. “Real-world tasks are unpredictable and human-centered. Robots need experience to adapt, and that comes from data.”
At CSAIL, Rus’s Distributed Robotics Lab has embraced this thinking. The team is building multimodal datasets of humans performing everyday tasks, from cooking and pouring to handing off objects. Rus said these recordings capture the subtleties of human action, from hand trajectories and joint torques to gaze and force interactions, providing a rich source of data for training AI systems.
The goal is not just to have robots replicate actions, but to enable them to generalize across tasks and adapt when conditions change.
In the kitchen testbed at CSAIL, for example, Rus’s team equips volunteers with sensors while they chop vegetables, pour liquids, and assemble meals. The sensors record not only joint and muscle movements but also subtle cues such as eye gaze, fingertip pressure, and object interactions.
AI models trained on this data can then perform the same tasks on robots with precision and robustness, learning how to recover when ingredients slip or tools misalign. These real-world datasets let researchers capture “long-tail” scenarios – rare but critical occurrences that model-based programming alone would miss.
Data at scale could transform manipulation
Tedrake discussed how scaling data transforms robot manipulation. His team has trained robots to perform dexterous tasks, such as slicing apples, observing diverse outcomes, and recovering from errors.
“Robots are now developing what looks like common sense for dexterous tasks,” he said. “It’s the same effect we’ve seen in language and vision: once you scale the data, surprising robustness emerges.”
In one example, he showed a bimanual robot equipped with simple grippers that learned to core and slice apples. Each apple differed slightly in size, firmness, or shape, yet the robot adapted automatically, adjusting grip and slicing motions based on prior experience.
Tedrake explained that, as the demonstration dataset expanded across multiple tasks, recovery behaviors—once manually programmed—began to emerge naturally, a sign that data can encode subtle, high-level common-sense knowledge about physical interactions.
Mathematical models come with a theoretical understanding
Kaelbling, who also spoke at the event, argued along with Billard and Park for the continuing importance of mathematical models, first principles, and theoretical understanding.
“Data can show us patterns, but models give us understanding,” Kaelbling said. “Without models, we risk systems that work, until they suddenly don’t. Safety-critical applications demand something deeper than trial-and-error learning.”
Billard said robotics differs fundamentally from vision or language: real-world data is scarce, simulations remain limited, and tasks involve infinite variability. While large datasets have propelled progress in perception and natural language understanding, she cautioned that blindly scaling data without an underlying structure risks creating brittle systems.
Park emphasized the richness of inductive biases from physics and biology—principles of motion, force, compliance, and hierarchical control—that data-driven methods alone cannot fully capture. He noted that carefully designed models can guide data collection and interpretation, helping ensure safety, efficiency, and robustness in complex tasks.
Finding middle ground
Garg, meanwhile, articulated the benefits of combining data-driven learning with structured models. He emphasized that while large datasets can reveal patterns and behaviors, models are necessary to generalize those insights and make them actionable.
“The best path forward may be a hybrid approach,” he said, “where we harness the scale of data while respecting the constraints and insights that models provide.”
Garg illustrated this with examples from collaborative manipulation tasks, where robots trained purely on raw data struggled with edge cases that a physics-informed model could anticipate.
The debate also drew historical parallels. Humanity has often acquired “know-how” before “know-why.” From sailing ships and internal combustion engines to airplanes and early computers, engineers relied on empirical observation long before fully understanding the underlying scientific principles.
Rus and Tedrake argued that modern robotics is following a similar trajectory: data allows robots to acquire practical experience in messy, unpredictable environments, while models provide the structure necessary to interpret and generalize that experience. This combination is essential, they said, to move from lab-bound experiments to robots capable of operating in homes, hospitals, and other real-world settings.
Diversity in thought is a strength in robotics
Throughout the debate, panelists emphasized the diversity of the robotics field itself. While deep learning has transformed perception and language tasks, robotics involves many challenges. These include high-dimensional control, variable human environments, interaction with deformable objects, and safety-critical constraints.
Tedrake noted that applying large pre-trained models from language directly to robots is insufficient; success requires multimodal learning and the integration of sensors that capture forces, motion, and tactile feedback.
Rus added that building large datasets across multiple robot platforms is crucial for generalization. “If we want robots to function across different homes, hospitals, or factories, we must capture the variety and unpredictability of the real world,” she said.
“Solving robotics is a long-term agenda,” Tedrake reflected. “It may take decades. But the debate itself is healthy. It means we’re testing our assumptions and sharpening our tools. The truth is, we’ll probably need both data and models – but which takes the lead, and when, remains unsettled.”
