Editor’s notice: This publish is a part of Into the Omniverse, a sequence centered on how builders, 3D practitioners and enterprises can remodel their workflows utilizing the most recent developments in OpenUSD and NVIDIA Omniverse.
Open supply has turn out to be important for driving innovation in robotics and autonomy. By offering entry to important infrastructure — from simulation frameworks to AI fashions — NVIDIA is enabling collaborative improvement that accelerates the trail to safer, extra succesful autonomous methods.
At CES earlier this month, NVIDIA launched a brand new suite of open bodily AI fashions and frameworks to speed up the event of humanoids, autonomous automobiles and different bodily AI embodiments. These instruments span your complete robotics improvement lifecycle — from high-fidelity world simulation and artificial knowledge technology to cloud-native orchestration and edge deployment — giving builders a modular toolkit to construct autonomous methods that may cause, study and act in the actual world.
OpenUSD offers the frequent framework that standardizes how 3D knowledge is shared throughout these bodily AI instruments, enabling builders to construct correct digital twins and reuse them seamlessly from simulation to deployment. NVIDIA Omniverse libraries, constructed on OpenUSD, function the supply of floor‑fact simulation that feeds your complete stack.
From Labs to the Present Ground
At CES 2026, builders introduced the NVIDIA bodily AI stack out of the lab and onto the present flooring, debuting machines starting from heavy gear and manufacturing unit assistants to social and repair robots.
The stack faucets into NVIDIA Cosmos world fashions; NVIDIA Isaac applied sciences, together with the brand new Isaac Lab-Enviornment open supply framework for coverage analysis; the NVIDIA Alpamayo open portfolio of AI fashions, simulation frameworks and bodily AI datasets for autonomous automobiles; and the NVIDIA OSMO framework to orchestrate coaching throughout compute environments.
Caterpillar’s Cat AI Assistant, powered by NVIDIA Nemotron open fashions for agentic AI and operating on the NVIDIA Jetson Thor edge AI module, brings pure language interplay straight into the cab of heavy automobiles. Operators can ask “Hey Cat”-style questions and get step‑by‑step steerage, in addition to alter security parameters by voice.
Behind the scenes, Caterpillar makes use of Omniverse libraries to construct manufacturing unit and job‑website digital twins that may assist simulate layouts, visitors patterns and multi‑machine workflows. These insights are fed again into gear and fleets earlier than adjustments are deployed to job websites, making AI‑assisted operations safer and extra environment friendly.
LEM Surgical showcased its Dynamis Robotic Surgical System, which is FDA-cleared and in routine medical use for spinal procedures. The following-generation system makes use of NVIDIA Jetson AGX Thor for compute, NVIDIA Holoscan for real-time sensor processing and NVIDIA Isaac for Healthcare to coach its autonomous arms.
LEM Surgical additionally makes use of NVIDIA Cosmos Switch — an open, totally customizable world mannequin that permits bodily based mostly artificial knowledge technology — to generate artificial coaching knowledge and the NVIDIA Isaac Sim framework for digital twin simulation. Designed as a dual-arm humanoid surgical robotic for hard-tissue surgical procedure, the Dynamis system mimics human surgeon dexterity and permits complicated spinal procedures with enhanced precision, assuaging strenuous bodily calls for on surgeons and surgical assistants.
NEURA Robotics is constructing cognitive robots on a full NVIDIA stack, utilizing Isaac Sim and Isaac Lab to coach its 4NE1 humanoid and MiPA service robots in OpenUSD‑based mostly digital twins earlier than deployment in home settings and workplaces. The corporate used NVIDIA Isaac GR00T‑Mimic to publish‑practice the Isaac GR00T basis mannequin for its platforms.
As well as, NEURA Robotics is collaborating with SAP and NVIDIA to combine SAP’s Joule brokers with its robots, utilizing the Mega NVIDIA Omniverse Blueprint to simulate and refine robotic habits in complicated, life like operational situations earlier than these brokers and behaviors are deployed into the corporate’s Neuraverse ecosystem, in addition to in actual‑world fleets.
AgiBot makes use of NVIDIA Cosmos Predict 2 because the world‑modeling spine for its Genie Envisioner (GE-Sim) platform — permitting the platform to generate motion‑conditioned movies grounded in sturdy visible and bodily priors. Combining this knowledge with Isaac Sim and Isaac Lab, in addition to publish‑coaching on AgiBot’s personal knowledge, lets insurance policies developed in Genie Envisioner switch extra reliably to Genie2 humanoids and compact Jetson Thor-powered tabletop robots.
Intbot is utilizing the NVIDIA Cosmos Cause 2 open mannequin to provide its social robots a “sixth sense” for the actual world — utilizing the mannequin’s reasoning capabilities to establish easy social cues and security context that transcend easy scripted duties. In its Cosmos Cookbook recipe, Intbot demonstrates how reasoning imaginative and prescient language fashions can support robots in deciding when to talk and how you can extra naturally work together with people.
How Robotics Builders Are Utilizing New Toolkits and Frameworks
NVIDIA lately launched Agile, an Isaac Lab-based engine for humanoid loco‑manipulation that packages a full, sim‑to‑actual‑verified workflow for coaching strong reinforcement studying insurance policies on platforms just like the Unitree G1 and LimX Dynamics TRON.
Robotics builders can use Agile’s constructed‑in activity configurations, Markov Choice Course of mathematical fashions for decision-making, coaching utilities and deterministic analysis instruments to tune insurance policies. Builders can then stress‑take a look at these insurance policies in Isaac Lab and switch locomotion and complete‑physique behaviors to real-world robots extra reliably and effectively.
Hugging Face and NVIDIA are bringing collectively their robotics communities by integrating NVIDIA Isaac GR00T N fashions and simulation frameworks into the LeRobot ecosystem. Builders can now entry Isaac GR00T N1.6 fashions and Isaac Lab‑Enviornment straight inside LeRobot to streamline coverage coaching and analysis.
Plus, Hugging Face’s open‑supply Reachy 2 humanoid is now totally interoperable with NVIDIA Jetson Thor, enabling the direct deployment of superior imaginative and prescient language motion (VLA) fashions for strong actual‑world efficiency.
ROBOTIS, a number one developer of good servos, industrial actuators, manipulators, open-source humanoid platforms and academic robotic kits, constructed an open supply sim-to-real pipeline utilizing NVIDIA Isaac applied sciences. The workflow begins with excessive‑constancy knowledge technology in Isaac Sim, scales up coaching units utilizing GR00T‑Mimic for augmentation after which advantageous‑tunes a VLA‑based mostly Isaac GR00T N mannequin that deploys on to {hardware} — accelerating the transition from simulation to strong actual‑world duties.
Get Plugged In
Study extra about OpenUSD and robotics improvement by exploring these sources:
- Learn this technical weblog to learn to develop generalist humanoid capabilities with NVIDIA Isaac and GR00T N1.6.
- Learn this technical weblog to learn to consider generalist robotic insurance policies in simulation utilizing NVIDIA Isaac Lab – Enviornment.
- Study how you can post-train Isaac GR00T with this two-part video tutorial.
- Watch NVIDIA founder and CEO Jensen Huang’s CES particular presentation.
- Enhance expertise for robotics improvement with the self-paced robotics studying path.
- Take part within the Cosmos Cookoff, a hands-on bodily AI problem the place builders use Cosmos Cause to energy robotics, autonomous methods and imaginative and prescient AI workflows.