Nvidia researchers have achieved a major leap in robotic dexterity thanks to Eureka, an AI agent that allegedly can teach bots complex skills like pen-spinning tricks as adroitly as humans.
The new technique, outlined in a paper published Thursday, builds on recent advances in large language models such as OpenAI’s GPT-4. Eureka leverages generative AI to autonomously write sophisticated reward algorithms that enable robots to learn via trial-and-error reinforcement learning. This approach has proven over 50% more effective than human-authored programs, the paper outlines.
“Eureka has also taught quadruped, dexterous hands, cobot arms and other robots to open drawers, use scissors, catch balls and nearly 30 different tasks,” an official blog post by Nvidia says.
Eureka is the latest demonstration of Nvidia’s pioneering work in steering AI with language models. Recently, the company open-sourced SteerLM—a method that aligns AI assistants to be more helpful by training them on human feedback.
Similar to Eureka, SteerLM also utilizes advances in language models, but focuses them on a different challenge—improving AI assistant alignment. SteerLM trains assistants by having them practice conversations, like a robot learning by doing. The system gives feedback on the assistant’s responses through attributes like helpfulness, humor, and quality.
For example, it’s like a robot learning to dance from videos labeled as good or bad, instead of having a human review thousands of random dances and selecting which ones are good or not (which is the way your typical AI chatbots are trained). By repeatedly practicing and getting feedback, the assistants learn to provide responses tailored to a user’s needs. This helps make AI more beneficial for real-world applications.
The common thread is the use of advanced neural networks in creative new ways, whether teaching robots or chatbots. Nvidia is pushing the boundaries on both hardware and software fronts.
For Eureka, the key was combining simulation technologies like the ones from Isaac Gym with the pattern-recognition prowess of language models. Eureka effectively “learns to learn,” optimizing its own reward algorithms over multiple training runs. It even accepts human input to refine its rewards.
This self-improving approach has proven highly generalizable so far, training robots of all kinds—legged, wheeled, flying and dexterous hands.
Nvidia’s Eureka and SteerLM are not just breaking barriers, they’re teaching robots and AI the art of finesse and insightful interaction. With every spin of a pen and witty chat, they’re sketching a future where AI doesn’t just mimic, but innovates alongside us.
