Interactive example app for mujoco-react — showcasing composable robot simulation with multiple robots, controllers, and debug tools.
| Robot | Controller | Features |
|---|---|---|
| Franka Panda | IK gizmo + gripper toggle (V) | 7-DOF arm, gizmo drag, click-to-select, graspable cubes |
| SO101 | Keyboard IK + gizmo | 2-link IK (WASD/QE), wrist (R/F, Z/C), gripper (V) |
| XLeRobot | Keyboard + dual arms | Mobile base (WASD), dual-arm IK, head pan/tilt, grippers (V/B) |
| Spot | Joint-hold showcase | Boston Dynamics Spot from MuJoCo Menagerie, debug overlays, drag/select |
| Unitree G1 | Free-base physics showcase | Unitree G1 humanoid from MuJoCo Menagerie, falls naturally, debug overlays, drag/select |
npm install
npm run devThe app demonstrates the composable children pattern — each feature is an independent R3F child you drop in or remove:
<MujocoProvider>
<MujocoCanvas config={entry.config} paused={sim.paused} speed={sim.speed}>
<OrbitControls />
<SceneRenderer />
{/* Opt-in IK controller plugin */}
<SceneChildren
robotKey={robotKey}
ikConfig={entry.hasIk ? entry.ikConfig : null}
showGizmo={sim.gizmo}
gizmoScale={entry.gizmoScale}
holdCtrl={entry.holdCtrl}
/>
<DragInteraction />
<ClickSelectOverlay />
{/* Per-robot controllers — swap in your own */}
{robotKey === 'franka' && <FrankaController />}
{robotKey === 'so101' && <SO101Controller />}
{robotKey === 'xlerobot' && <XLeRobotController />}
{entry.holdCtrl && <HoldCtrl values={entry.holdCtrl} />}
{/* Debug overlays */}
<ContactMarkers visible={debug.contacts} />
<Debug
showSites={debug.sites}
showJoints={debug.joints}
showCameras={debug.cameras}
/>
{/* Scene decoration */}
<ambientLight />
<directionalLight castShadow />
</MujocoCanvas>
</MujocoProvider>Controllers are thin wrappers around library hooks. Each robot's controller is a static config object passed to a generic hook — no per-robot logic.
The simplest possible controller — one useKeyboardTeleop call:
import { RobotActuators, useKeyboardTeleop } from 'mujoco-react';
export function FrankaController() {
useKeyboardTeleop({
bindings: { v: { actuator: RobotActuators.franka.gripper, toggle: [0, 255] } },
});
return null;
}Arm positioning comes from <IkGizmo /> via useIkController(ikConfig) — the controller only adds the gripper toggle.
Pure config objects passed to useArmController:
import { useArmController } from './useArmController';
const config: ArmControllerConfig = {
numActuators: 6,
arms: [{
indices: [0, 1, 2, 3, 4, 5],
keys: ['KeyD', 'KeyA', 'KeyW', 'KeyS', 'KeyQ', 'KeyE',
'KeyR', 'KeyF', 'KeyZ', 'KeyC', 'KeyV'],
initialJoints: [0.0158, 2.052, 2.1307, -0.0845, 1.5857, -0.3745],
}],
};
export function SO101Controller() {
useArmController(config);
return null;
}Generic hook for keyboard-driven arm control with 2-link IK. Accepts a static config:
interface ArmControllerConfig {
numActuators: number;
base?: BaseConfig; // Mobile base (WASD drive)
arms: ArmConfig[]; // One or more arms with IK
head?: HeadConfig; // Pan/tilt head
}
interface ArmConfig {
indices: number[]; // Actuator indices
keys: string[]; // 11 key bindings (rotation, EE, pitch, roll, gripper)
initialJoints?: number[]; // Start position (uses FK to sync IK state)
initialRotation?: number;
initialRoll?: number;
}Features:
- Gizmo coexistence — uses
useIk({ optional: true })to detect if an IkController is active. When arm keys are pressed, automatically disables the library's IK and syncs from the current arm position so there's no jump - Per-arm IK — each arm has independent 2-link IK state
- Config-driven — swap robots by changing the config object, no code changes
A hook + component pattern:
function ClickSelectOverlay() {
const selectedBodyId = useClickSelect(); // double-click raycasting hook
return <SelectionHighlight bodyId={selectedBodyId} />;
}For objects that need to be picked up by grippers, set these MuJoCo contact parameters:
sceneObjects: [{
name: 'cube',
type: 'box',
size: [0.025, 0.025, 0.025],
position: [0.4, 0, 0.025],
rgba: [0.9, 0.2, 0.15, 1],
mass: 0.05,
freejoint: true,
friction: '1.5 0.3 0.1', // high static friction
solref: '0.01 1', // stiff contact solver
solimp: '0.95 0.99 0.001 0.5 2', // tight impedance
condim: 4, // 4D contact (friction cone)
}]Without condim: 4 and high friction, blocks will slide out of the gripper when lifted.
The Leva panel provides runtime controls:
- Robot — switch between Franka, SO101, XLeRobot, Spot, Unitree G1
- Simulation — pause, speed, gravity compensation, IK gizmo toggle, reset
- Debug — contacts, sites, joints, and camera frustum visualization
| Key | Franka | SO101 | XLeRobot | Spot / G1 |
|---|---|---|---|---|
| WASD | -- | EE forward/back/up/down | Base drive | -- |
| Q/E | -- | EE left/right | -- | -- |
| R/F | -- | Wrist pitch | Head pan | -- |
| Z/C | -- | Wrist roll | -- | -- |
| V | Gripper toggle | Gripper toggle | Left gripper | -- |
| B | -- | -- | Right gripper | -- |
| 7-0, Y/U/I/O | -- | -- | Left arm | -- |
| H-L, N-/ | -- | -- | Right arm | -- |
| Double-click | Select body | Select body | Select body | Select body |
npm run build # Production build
npx tsc --noEmit # Type checkOpen /?demo=engine (or the MuJoCo engine demos link) to exercise analytic
ball-joint IK, a three-input PID servo, a scalar actuator after multi-input
actuators, indexed cloth rendering, pause/reset, and complete state snapshots.
The existing robot playground retains its model selector, IK gizmos, flight
controller, and mounted camera streams.
The example consumes the published mujoco-react package. Release validation also
links the local library with npm link mujoco-react before installing the release.
The engine demo links to /?demo=speculative for a fast sphere/thin wall comparison and /?demo=plastic for elastic versus yielding springs with full-state save/restore. These scenes use the MuJoCo 4.0.1 engine bundled with mujoco-react.
Open /?demo=runtime for exact worker stepping, cancellation, reset, full snapshots,
and three synchronized camera observations. Three live GPU camera panes remain
interactive during long physics batches. HUD updates intentionally pass an inline
scene config to exercise stable model lifetime. UI FPS and simulation throughput
are shown separately. This demonstrates simulator mechanics, not a learned policy.
XLeRobot exposes one IK gizmo per gripper. The IK gizmo toggle shows both; dragging one controls only that arm. Keyboard arm movement temporarily takes over that arm’s IK controller, while base, head, and gripper controls remain independent.
Robot selection is reflected in the URL (for example ?robot=xlerobot or
?robot=so101OverheadBimanual). Reload or share that URL to open the same robot;
unknown robot names fall back to Franka. Other query parameters are preserved.
Open ?demo=rgbd for aligned RGB/depth previews and GPU regression checks.
The named MuJoCo camera sees reference box fronts at 2 m and 3 m, independent
of the interactive viewer. Mirrored capture flips both arrays together.