The test begins when the robot leaves simulation
A simulated robot can complete thousands of successful attempts while depending on a contact model or observation that is unavailable on the real machine. The apparent success then measures the simulated task, not the deployed one. Transfer must be evaluated against the physical robot, its sensors and the actual operating conditions.
Sim-to-real is the work of making policies or models trained in simulation useful on hardware. It includes perception but also control, contact, timing and calibration. This guide focuses on behavior transfer. Synthetic perception data addresses the narrower question of creating training observations and measuring their value.
Randomization is a tool, not a guarantee
Tobin and colleagues demonstrated domain randomization for a particular localization problem. That supports a method for reducing dependence on simulated appearance. It does not settle contact dynamics or establish successful transfer for every robot task.
NVIDIA's Isaac Lab documentation describes a framework for robot learning. Its tutorial connects task configuration, calibration and deployment. These sources establish research and implementation approaches. Tool availability does not independently establish a customer's success rate, operating cost or safety case.
Model the errors that change the decision
An inaccurate friction coefficient matters when it changes whether an object slips; an inaccurate background texture may be irrelevant to a controller using other observations. Modeling effort should follow sensitivity to the task. Increasing detail everywhere can consume compute and engineering time without improving the behavior that matters.
Latency and calibration deserve the same attention as visible geometry. A controller can receive accurate observations too late, or command a joint through a mapping that differs from the simulated one. A transfer report should identify these interfaces and explain how failures are detected and recovered, rather than showing only successful demonstrations.
Validate beyond a favorable average
Suppose a hypothetical policy succeeds in 95 of 100 physical attempts. That observed result does not establish performance in all conditions, and the five failures need examination. Repeating an easy starting state gives different evidence from testing a representative range of objects, disturbances and recovery situations.
These counts are illustrative. We would report the test design, the definition of success and the distribution of failure types before relying on one percentage. Validation should also consider what an unsuccessful attempt costs and whether it can be recovered safely. A policy can have a high average score while failing in an unacceptable way.
The economic benefit is a changed development process
Simulation is economically useful when it reduces the time or cost needed to reach acceptable real performance, or enables work that would otherwise be impractical. More simulated episodes alone do not establish that benefit. Building environments, checking models and maintaining the transfer pipeline are part of the investment.
The synthetic-data economics guide is consolidated into the perception-data article. We retain this separate guide because validating a physical policy is a different reader task. A simulator purchase can support both workflows, but that does not make their evaluation criteria interchangeable.
What would change our judgment
The strongest evidence follows a defined task from simulation through repeated physical tests and explains which changes improved transfer. Independent evaluation, disclosed failures and a comparison with a simpler development process would make the case more persuasive than additional rendered demonstrations.
The case weakens when performance requires repeated task-specific repairs, when simulation improvements do not survive on hardware, or when validation consumes the savings attributed to training. Our conclusion is that transfer is a measured engineering outcome.