Cornell's RHyME Enables Robots to Learn from Videos

Cornell University researchers have developed a new robotic framework called RHyME—short for Retrieval for Hybrid Imitation under Mismatched Execution—that allows robots to learn tasks by watching a single how-to video. The system uses artificial intelligence to translate human demonstrations into actions that robots can follow. By reducing the need for extensive, step-by-step task instructions, the framework could decrease the training time, energy, and money needed to deploy robotic systems.

The research team, led by doctoral student Kushal Kedia and assistant professor Sanjiban Choudhury, is using what is known as imitation learning. This approach enables robots to learn a sequence of tasks more quickly and helps the machines adapt to real-world environments. The team compared the AI translation process to converting between languages, saying it is similar to translating French to English by converting human demonstrations into robotic actions.

The development comes as part of efforts to improve home robot assistants and other robotic applications, addressing challenges faced when robots lose tools or screws during tasks. Further details on the framework and its implementation are available through Cornell University's publications.



Source: Cornell University

Source Type: College

Source Location: Ithaca

Apr 23, 2025 at 8:30 AM College

Schweizer Marks 200th Goal, Ells Earn Honors

Finn Schweizer and Paige Ells earn Twin Tiers athletic honors.

Apr 23, 2025 at 12:20 PM Schools

Cornell’s BOOM Showcases Student Tech Innovations

Cornell hosts free BOOM event showcasing innovative student tech projects.

Apr 23, 2025 at 12:01 PM College

Cornell Warns: Warm Weather Fuels Tick Risks

Experts warn increased outdoor activity raises tick exposure risk.

Apr 22, 2025 at 3:07 PM College

Cornell Campus: Hill Shines in Every Season

Cornell campus shows impressive, year-round beauty and vibrant landscapes.

Apr 22, 2025 at 8:30 AM College

Cornell Study: Dogs Offer Insights on ACL Injury

Dogs' ACL injuries model human osteoarthritis, promising improved treatment insights.

Apr 21, 2025 at 2:37 PM College

Cornell Astronomer Guides Historic Mars Rover Mission

Cornell research drove Mars Rover's prolific mission and groundbreaking planetary insights.

Apr 21, 2025 at 8:30 AM College

Ithaca Vineyard Closes Easter, Reopens Monday

Six Mile Creek Vineyard closes on Easter; resumes business Monday.

Apr 20, 2025 at 9:36 AM winery

Cornell Finds Dream Study Spot at Arboretum

Cornell finds an ideal, quiet study spot at the campus Arboretum.

Apr 18, 2025 at 2:30 PM College

CCE Expands Childcare Options in Jefferson County

CCE in Jefferson County offers safe, affordable afterschool childcare that boosts academics.

Apr 18, 2025 at 8:30 AM College