Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

mujoco-react example

Interactive example app for mujoco-react — showcasing composable robot simulation with multiple robots, controllers, and debug tools.

Robots

Robot Controller Features
Franka Panda IK gizmo + gripper toggle (V) 7-DOF arm, gizmo drag, click-to-select, graspable cubes
SO101 Keyboard IK + gizmo 2-link IK (WASD/QE), wrist (R/F, Z/C), gripper (V)
XLeRobot Keyboard + dual arms Mobile base (WASD), dual-arm IK, head pan/tilt, grippers (V/B)
Spot Joint-hold showcase Boston Dynamics Spot from MuJoCo Menagerie, debug overlays, drag/select
Unitree G1 Free-base physics showcase Unitree G1 humanoid from MuJoCo Menagerie, falls naturally, debug overlays, drag/select

Getting Started

npm install
npm run dev

Architecture

The app demonstrates the composable children pattern — each feature is an independent R3F child you drop in or remove:

<MujocoProvider>
  <MujocoCanvas config={entry.config} paused={sim.paused} speed={sim.speed}>
    <OrbitControls />
    <SceneRenderer />

    {/* Opt-in IK controller plugin */}
    <SceneChildren
      robotKey={robotKey}
      ikConfig={entry.hasIk ? entry.ikConfig : null}
      showGizmo={sim.gizmo}
      gizmoScale={entry.gizmoScale}
      holdCtrl={entry.holdCtrl}
    />
    <DragInteraction />
    <ClickSelectOverlay />

    {/* Per-robot controllers — swap in your own */}
    {robotKey === 'franka' && <FrankaController />}
    {robotKey === 'so101' && <SO101Controller />}
    {robotKey === 'xlerobot' && <XLeRobotController />}
    {entry.holdCtrl && <HoldCtrl values={entry.holdCtrl} />}

    {/* Debug overlays */}
    <ContactMarkers visible={debug.contacts} />
    <Debug
      showSites={debug.sites}
      showJoints={debug.joints}
      showCameras={debug.cameras}
    />

    {/* Scene decoration */}
    <ambientLight />
    <directionalLight castShadow />
  </MujocoCanvas>
</MujocoProvider>

Controllers

Controllers are thin wrappers around library hooks. Each robot's controller is a static config object passed to a generic hook — no per-robot logic.

FrankaController

The simplest possible controller — one useKeyboardTeleop call:

import { RobotActuators, useKeyboardTeleop } from 'mujoco-react';

export function FrankaController() {
  useKeyboardTeleop({
    bindings: { v: { actuator: RobotActuators.franka.gripper, toggle: [0, 255] } },
  });
  return null;
}

Arm positioning comes from <IkGizmo /> via useIkController(ikConfig) — the controller only adds the gripper toggle.

SO101Controller / XLeRobotController

Pure config objects passed to useArmController:

import { useArmController } from './useArmController';

const config: ArmControllerConfig = {
  numActuators: 6,
  arms: [{
    indices: [0, 1, 2, 3, 4, 5],
    keys: ['KeyD', 'KeyA', 'KeyW', 'KeyS', 'KeyQ', 'KeyE',
           'KeyR', 'KeyF', 'KeyZ', 'KeyC', 'KeyV'],
    initialJoints: [0.0158, 2.052, 2.1307, -0.0845, 1.5857, -0.3745],
  }],
};

export function SO101Controller() {
  useArmController(config);
  return null;
}

useArmController

Generic hook for keyboard-driven arm control with 2-link IK. Accepts a static config:

interface ArmControllerConfig {
  numActuators: number;
  base?: BaseConfig;     // Mobile base (WASD drive)
  arms: ArmConfig[];     // One or more arms with IK
  head?: HeadConfig;     // Pan/tilt head
}

interface ArmConfig {
  indices: number[];       // Actuator indices
  keys: string[];          // 11 key bindings (rotation, EE, pitch, roll, gripper)
  initialJoints?: number[];  // Start position (uses FK to sync IK state)
  initialRotation?: number;
  initialRoll?: number;
}

Features:

  • Gizmo coexistence — uses useIk({ optional: true }) to detect if an IkController is active. When arm keys are pressed, automatically disables the library's IK and syncs from the current arm position so there's no jump
  • Per-arm IK — each arm has independent 2-link IK state
  • Config-driven — swap robots by changing the config object, no code changes

Click-to-Select

A hook + component pattern:

function ClickSelectOverlay() {
  const selectedBodyId = useClickSelect(); // double-click raycasting hook
  return <SelectionHighlight bodyId={selectedBodyId} />;
}

Graspable Objects

For objects that need to be picked up by grippers, set these MuJoCo contact parameters:

sceneObjects: [{
  name: 'cube',
  type: 'box',
  size: [0.025, 0.025, 0.025],
  position: [0.4, 0, 0.025],
  rgba: [0.9, 0.2, 0.15, 1],
  mass: 0.05,
  freejoint: true,
  friction: '1.5 0.3 0.1',            // high static friction
  solref: '0.01 1',                    // stiff contact solver
  solimp: '0.95 0.99 0.001 0.5 2',    // tight impedance
  condim: 4,                           // 4D contact (friction cone)
}]

Without condim: 4 and high friction, blocks will slide out of the gripper when lifted.

Control Panel

The Leva panel provides runtime controls:

  • Robot — switch between Franka, SO101, XLeRobot, Spot, Unitree G1
  • Simulation — pause, speed, gravity compensation, IK gizmo toggle, reset
  • Debug — contacts, sites, joints, and camera frustum visualization

Key Bindings

Key Franka SO101 XLeRobot Spot / G1
WASD -- EE forward/back/up/down Base drive --
Q/E -- EE left/right -- --
R/F -- Wrist pitch Head pan --
Z/C -- Wrist roll -- --
V Gripper toggle Gripper toggle Left gripper --
B -- -- Right gripper --
7-0, Y/U/I/O -- -- Left arm --
H-L, N-/ -- -- Right arm --
Double-click Select body Select body Select body Select body

Building

npm run build     # Production build
npx tsc --noEmit  # Type check

Engine feature demo

Open /?demo=engine (or the MuJoCo engine demos link) to exercise analytic ball-joint IK, a three-input PID servo, a scalar actuator after multi-input actuators, indexed cloth rendering, pause/reset, and complete state snapshots. The existing robot playground retains its model selector, IK gizmos, flight controller, and mounted camera streams.

The example consumes the published mujoco-react package. Release validation also links the local library with npm link mujoco-react before installing the release.

The engine demo links to /?demo=speculative for a fast sphere/thin wall comparison and /?demo=plastic for elastic versus yielding springs with full-state save/restore. These scenes use the MuJoCo 4.0.1 engine bundled with mujoco-react.

Worker simulation demo

Open /?demo=runtime for exact worker stepping, cancellation, reset, full snapshots, and three synchronized camera observations. Three live GPU camera panes remain interactive during long physics batches. HUD updates intentionally pass an inline scene config to exercise stable model lifetime. UI FPS and simulation throughput are shown separately. This demonstrates simulator mechanics, not a learned policy.

XLeRobot exposes one IK gizmo per gripper. The IK gizmo toggle shows both; dragging one controls only that arm. Keyboard arm movement temporarily takes over that arm’s IK controller, while base, head, and gripper controls remain independent.

Robot selection is reflected in the URL (for example ?robot=xlerobot or ?robot=so101OverheadBimanual). Reload or share that URL to open the same robot; unknown robot names fall back to Franka. Other query parameters are preserved.

Metric RGB-D capture

Open ?demo=rgbd for aligned RGB/depth previews and GPU regression checks. The named MuJoCo camera sees reference box fronts at 2 m and 3 m, independent of the interactive viewer. Mirrored capture flips both arrays together.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages