Sim-to-real gap

Extracted Attributes

• Field

  Robotics and Physical Artificial Intelligence

• Definition

  Performance degradation occurring when a policy trained in simulation is deployed on real physical hardware

• Primary Causes

  Sensory noise, actuator dynamics, contact physics approximations, and unmodeled environmental factors

• Mitigation Techniques

  Domain randomization, teleoperation, cross-embodiment learning, world modeling, and system identification

Timeline

• Niantic Spatial forms as a spin-off from Niantic, Inc. to develop geospatial world models for robotics, defense, and utilities. (Source: Wikipedia)

  2025-05-01

• 1X plans to launch NEO, a bipedal robot leveraging world models designed to overcome the sim-to-real gap. (Source: Document c4c99c45-ab82-495f-b76a-fc9ae670bc0d)

  2026-01-01

Niantic Spatial

Niantic Spatial, Inc. (ny-AN-tik SPAY-shəl) is an American-based physical artificial intelligence (AI) and spatial computing company headquartered in San Francisco. The company develops geospatial world models that gives machines understanding of the physical world through 3D reconstruction, visual positioning, and semantic understanding. They support enterprise customers in robotics, defense, and utilities. The company was formed in May 2025 as a spin-off from Niantic, Inc., following the sale of Niantic's licensed games to Scopely (now operating as Scopely Explore). Niantic Spatial retained its original franchises Ingress and Peridot alongside its augmented reality and geospatial technology.

Web Search Results

• Sim-to-Real Gap in Robotics — PatSnap Eureka | Patsnap

  The sim-to-real gap refers to the performance degradation that occurs when a manipulation policy trained in a physics simulator is deployed on a real robot. Differences in contact dynamics, sensor noise, actuator response, and visual appearance between simulation and reality cause policies that work well in simulation to fail in the real world. Domain randomization is a technique where simulation parameters — such as object mass, friction coefficients, lighting, texture, and actuator dynamics — are varied randomly during training. The goal is to make the learned policy robust enough that real-world conditions fall within the distribution of simulated conditions the policy has already encountered. [...] ## What Is the Sim-to-Real Gap in Robotic Manipulation? The sim-to-real gap refers to the performance degradation that occurs when a manipulation policy trained in a physics simulator is deployed on a real robot. Policies that achieve near-perfect task completion in simulation frequently fail when exposed to real-world hardware, surfaces, and sensor noise — because the simulator cannot perfectly replicate physical reality. [...] # Bridging the Sim-to-Real Gap in Robotic Manipulation The sim-to-real gap is one of the central engineering challenges in deploying autonomous manipulation systems. Discover how robotics engineers use domain randomization, system identification, and adaptive policy architectures to transfer learned behaviours from simulation to the physical world — and how PatSnap Eureka accelerates that research. ## What Is the Sim-to-Real Gap in Robotic Manipulation?

• Sim-to-Real Gap in Robotics — Definition & Data | Claru

  ## What Is Sim-to-Real Gap? The sim-to-real gap refers to the systematic performance degradation that occurs when a robot control policy, trained entirely in a simulated environment, is deployed on physical hardware in the real world. This gap exists because no simulator perfectly reproduces the visual appearance, physics, sensor characteristics, and actuator dynamics of the real world. A policy that achieves 95% success rate in simulation may achieve only 30-60% on the same task with a physical robot. [...] # Sim-to-Real Gap: Why Simulated Robot Policies Fail on Real Hardware The sim-to-real gap is the performance degradation that occurs when a robot policy trained entirely in simulation is deployed on physical hardware. Differences in visual rendering, contact physics, sensor noise, and actuator dynamics between simulator and real world cause policies to encounter conditions they never saw during training, leading to failures that range from reduced accuracy to complete task breakdown. ## What Is Sim-to-Real Gap? [...] Understanding the sim-to-real gap is essential for any team that uses simulation as part of their robot learning pipeline. The gap determines how much real-world data is needed to achieve production-level performance, which simulation parameters to randomize during training, and what data collection protocols to implement for real-world fine-tuning. ## Historical Context

• Sim-to-real transfer and reality gap modeling in model predictive control for autonomous driving | Applied Intelligence | Springer Nature Link

  This makes the MPC design process benefit greatly from the use of simulation environments. Autonomous driving simulators are playing an increasingly important role in the development and testing of systems in autonomous vehicles, since they allow massive automated validation and testing in any type of scenario, as well as parameter refinement without physical or real-time constraints. Nevertheless, this powerful methodology is only applicable when the discrepancy between simulated and real environments is reasonably low and is thoroughly narrowed. This discrepancy is usually referred to as sim-to-real gap (simulation- to-reality) or reality gap. [...] The paper aims to evaluate the reality gap of a steering control system of an autonomous vehicle due to sim-to-real transfer done in design and implementation, with a methodology that compares the controller’s performance in a real environment and its virtual equivalent. The driving simulator used in this work does not include a Citröen C4 vehicle in its collection; so a vehicle with similar characteristics (Toyota Prius) was used, parameterizing its localization device with a precision of 2 cm, such as the real car counterpart. The real and simulated car geometry was based on the center of gravity and is further described in Section 3.2. Standard road features were used in both real and simulated scenarios. The vector [xw,yw,v,ψ]T defines the state vector of the system, where xw is the [...] With regards to reality gap, uniform tests show more similaritiy than nob-uniform ones, for both indicators. Thus the sim-to-real transfer is better when a non-uniform distribution is used. More than 91% and 71% of similarity is measured with PCC and MNCC on uniform tests, respectively. The mean percentage of similarity decreases with non-uniform tentatives in both indicators to 70% and 65%. So, both indicators depict a similarity decrement when a non-uniform tentative distribution setup is compared against uniform distributions. When dynamics are more demanding, the bycicle kinematic model and some CARLA features yield a poor representation of reality. Applying new sim-to-real transfer methods will be a solution to reduce the reality gap in this setup.

• What Is Sim-to-Real? — Train an SO-101 Robot From Sim-to-Real With NVIDIA Isaac

  Warning The sim-to-real gap is often larger than expected. And while colloquially we may discuss “the gap” as if it’s a single entity, the gap is a complex combination of gaps in sensing, actuation, physics, and modeling. Never assume a policy will “just work” on real hardware without systematic testing and iteration. ## Sources of the Gap# ### Sensing Gaps# Camera models lack real sensor noise, blur, and distortion Depth sensors have idealized measurements without artifacts Simulated lighting differs from real lighting conditions ### Actuation Gaps# Motor models lack friction, backlash, and thermal effects Joint dynamics are simplified Control loop timing differs between simulation and hardware ### Physics Gaps# Contact dynamics (friction, restitution) are approximations [...] Identify the four major categories of sim-to-real gaps Explain why transfer is difficult even with high-fidelity simulation ## Sim-to-Real Defined# Sim-to-real refers to the process of training a policy in simulation and deploying it on real hardware. The goal is a policy that performs well in the real world despite being trained entirely (or primarily) in simulation. Sim-to-Real Sim-to-Real with Unitree H1# ## The Sim-to-Real Gap# The sim-to-real gap is the performance difference between simulation and reality. A policy achieving high success rates in simulation may perform significantly worse on real hardware. Warning [...] Deformable objects are difficult to simulate accurately Fluid dynamics and granular materials are computationally expensive ### Modeling Gaps# CAD models differ from as-built hardware Mass and inertia properties are estimates ## What Makes Transfer Hard?# The sim-to-real gap isn’t just about simulation fidelity. Even with perfect simulation, transfer is challenging because: Distribution shift: Real-world conditions vary from training Compounding errors: Small perception errors lead to large action errors Unmodeled dynamics: Real physics has effects that may not be represented in simulation Temporal differences: Real-time constraints affect behavior ## Summary#

• [PDF] What Went Wrong? Closing the Sim-to-Real Gap via Differentiable ...

  Yet, robot simulators inevitably exhibit disparities from the real-world dynamics, yielding inaccuracies that manifest as the dynamical simulation-to-reality (sim-to-real) gap. Existing literature has proposed to close this gap by actively modifying specific simulator parameters to align the simulated data with real-world obser-vations. However, the set of tunable parameters is usually manually selected to reduce the search space in a case-by-case manner, which is hard to scale up for complex systems and requires extensive domain knowledge. To address the scal-ability issue and automate the parameter-tuning process, we introduce COMPASS, which aligns the simulator with the real world by discovering the causal relation-ship between the environment parameters and the sim-to-real gap. [...] between the environment parameters and the sim-to-real gap. Concretely, our method learns a differentiable mapping from the environment parameters to the differences between simulated and real-world robot-object trajectories. This mapping is governed by a simultaneously learned causal graph to help prune the search space of parameters, provide better interpretability, and improve general-ization on unseen parameters. We perform experiments to achieve both sim-to-sim and sim-to-real transfer, and show that our method has significant improve-ments in trajectory alignment and task success rate over strong baselines in sev-eral challenging manipulation tasks. Demos are available on our project website: [...] In our work, we assume that the simulator’s system dynamics are conditioned on environment pa-rameters ϵ ∈R|E|, i.e., P : S × A × S × R|E| →R+, where E is the set of all tunable environment parameters and | · | measures the cardinality of the set. Given a simulator parameterized by ϵ, the agent is optimizing Eπθ,ϵ[R(τ)]. When the simulation dynamics are very close to the real-world dy-namics, one can expect the trajectory rollouts in the simulator to be close to that in the real world as well. Hence, an optimal agent trained in the simulator would expect near-optimal performance in the real world . However, due to unmodeled dynamics and inaccurate environment parameters, the 3 simulation dynamics are different from the real world (i.e., there exists a sim-to-real gap), resulting in