Skip to content
googlePublic

About

USD Hydra delegate for Mitsuba 3

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

90 stars

Watchers

4 watching

Forks

hdMitsuba - USD Hydra Delegate for Mitsuba 3

Linux CI macOS CI

hdMitsuba is a USD Hydra render delegate that enables OpenUSD to render scenes using Mitsuba 3. It allows integrating Mitsuba rendering into USD-compatible applications (e.g., usdview, Houdini, ...), or using it programmatically via Hydra.

Mitsuba in usdview

The video shows interactive rendering using Mitsuba in usdview on MacOS (using Metal GPU acceleration, and with experimental kernel freezing enabled).

Motivation

This Hydra delegate bridges the gap between rendering research and production pipelines by bringing Mitsuba 3 into the Pixar OpenUSD ecosystem. This allows to use the wide range of existing USD tools for scene assembly, while benefitting from various Mitsuba capabilities. USD provides a clean separation between renderer-agnostic scene assembly and final image rendering. Some of the key benefits of hdMitsuba are:

  • Direct USD rendering: Render USD stages directly using Mitsuba 3 without (inefficient) manual scene translation. This allows harnessing both USD-compatible tools as well as USD's APIs for authoring scenes that are then rendered in Mitsuba. Moreover, Mitsuba can be used as a viewport renderer in USD-native applications.
  • Efficient incremental updates: Leverage Hydra's incremental update mechanism. Only modified scene elements (e.g., camera motion, material edits) are passed to Mitsuba, enabling efficient interactive and sequence rendering.
  • Advanced research features: Exposes Mitsuba's unique capabilities, such as spectral and polarized rendering, and ease of research prototyping, to the USD ecosystem.
  • Extensibility: Mitsuba's modular design allows to easily prototype custom BSDFs or camera models (often in Python), and immediately use them within USD pipelines.

Components

This repository provies the following components:

  • hdmitsuba: C++ implementation of the Hydra render delegate and its unit tests.
  • usd_mitsuba: A Python package for offline USD-to-Mitsuba scene translation. This code provides a reference implementation converting a USD scene to a Mitsuba scene dictionary. This is much slower than the Hydra delegate, but offers a very hackable and easy-to-understand code path. Morever, we use this code as a reference to test the hydra delegate.
  • render_engine: A simple USD render engine with Python bindings. This allows rendering using Hydra delegates and directly obtaining the result as NumPy arrays.
  • nanobind/usd.h: A generic Nanobind caster for USD types. This allows writing Nanobind-wrapped functions that interact with USD's Python bindings.

Features & Documentation

hdMitsuba supports many features of both USD (subdivision surfaces, displacement maps, skeletal animation, instances, ...) and Mitsuba (native BSDFs, integrators, ...). For a detailed guide on supported features, custom render settings (variants, sample counts), and scene authoring examples, please refer to the hdMitsuba User Guide. Please also see the extensive unit tests in hdmitsuba/tests/ for usage examples.

Building

Prerequisites

  1. Toolchain: A modern C++17 compiler (e.g., Clang, GCC)
    • macOS: Clang is required (Xcode toolchain).
    • Linux: Either GCC or Clang work. If you install OpenUSD using conda, you need to use GCC for ABI compatibility.
  2. Dependencies:
    • OpenUSD: For example built locally or installed via conda. Due to the size and complexity of this dependency, we currently do not include it as a git submodule.
    • Mitsuba 3: Locally built from source. Similarly to USD, it is not included as a submodule. For compatibility, it is recommended to use commit aa01500f (which is the version validated in our CI configuration). The pip-installed version of Mitsuba currently is not supported.
    • Additional dependencies: Additional C++ dependencies are included as git submoduls. When cloning this repository, use git clone --recursive to initialize submodules. Otherwise, they can also be initialized using git submodule update --init --recursive.
    • Python dependencies: numpy, pytest.

Build Steps

The build configuration has been tested on Linux and MacOS. Configure and build using CMake and Ninja from the repository root:

cmake -G Ninja \
  -DCMAKE_PREFIX_PATH="/path/to/usd/installation/prefix" \
  -DMITSUBA_DIR="/path/to/mitsuba3/source" \
  -B build
cmake --build build

To build against a local Houdini installation (Houdini 22.0 or newer, which bundles OpenUSD >= 26.05 and Python), pass the Houdini installation path as CMAKE_PREFIX_PATH (e.g. /Applications/Houdini/Current on macOS, or /opt/hfs22.0 on Linux):

cmake -G Ninja \
  -DCMAKE_PREFIX_PATH="/Applications/Houdini/Current" \
  -DMITSUBA_DIR="/path/to/mitsuba3/source" \
  -B build
cmake --build build

Running Tests

The built binaries locate the USD and Mitsuba libraries through their RPATH (which refers to the USD installation and the Mitsuba build directory), so LD_LIBRARY_PATH does not need to be set. Only two things need to be configured:

  • PXR_PLUGINPATH_NAME must contain the directory of the hdMitsuba plugin (build/hdmitsuba, or <prefix>/plugin/usd after cmake --install build --prefix <prefix>). This is all that is needed to use the delegate in an existing USD application such as usdview: in a Python process, the delegate itself makes the Python modules of the Mitsuba build importable.
  • PYTHONPATH must additionally contain the Python modules of this project, Mitsuba and USD to run the tests, the render CLI or the offline translator.

Environment Setup

CMake generates a setpath.sh script in your build directory that automatically configures these environment variables based on your build configuration.

Before running any tests, source this script from the repository root:

source build/setpath.sh

Running Test Suites

1. Python Tests (pytest)

We use pytest to run all Python-based tests (including high-level integration tests, Hydra delegate tests, and offline translator tests). Ensure you have sourced setpath.sh first, and then run:

# Run all Python tests
pytest

# Run the more extensive suite of state update tests (not enabled by default):
pytest hdmitsuba/tests/fuzzing

2. C++ Unit Tests (ctest)

We use ctest (part of CMake) to run the C++ unit tests. These tests verify some of the internal Hydra delegate classes and geometry processing helpers.

Sourcing setpath.sh is generally not required for C++ tests as CMake configures build-tree RPATHs.

ctest --test-dir build

3. Running with usdview

You can visualize USD stages interactively in Pixar's usdview using the Mitsuba delegate:

# Launch usdview with Mitsuba pre-selected as the renderer
usdview -r Mitsuba /path/to/scene.usd

Or, launch usdview normally and select Mitsuba from the Renderer menu in the top menu bar.

4. Running the Render CLI

You can use the provided command-line script to render a USD stage using hdMitsuba programmatically:

# Render a scene using the default Mitsuba variant
python render_engine/render_scene_cli.py --scene /path/to/scene.usd

# Render with a specific Mitsuba variant, sample count, and output path
python render_engine/render_scene_cli.py --scene /path/to/scene.usd --variant llvm_ad_rgb --spp 32 --output output.png

# Render an animated scene at a specific frame (time code)
python render_engine/render_scene_cli.py --scene /path/to/scene.usd --frame 1030

Key Features of the CLI:

  • Automatic Camera Framing: If your USD stage has no authored cameras, the CLI automatically computes the scene's bounding box and creates a default camera (/default_cam) that frames the entire scene.
  • Time Code Selection: Pass --frame <time code> to render an animated stage at a specific time code, or --frame default to render at UsdTimeCode::Default(). If --frame is omitted, the CLI renders at the stage's start time code if the stage has an authored time code range (startTimeCode/endTimeCode), and at the default time code otherwise.
  • Performance Profiling: Pass the --profile flag to collect a detailed Chrome tracing JSON profile (saved as <output_path>.trace.json). You can load this file into Perfetto UI or chrome://tracing to inspect and profile execution times (JIT tracing, scene sync, rendering).
  • Dr.Jit Logging: Pass --drjit_log info to print Dr.Jit's own log statements.

5. Running with Houdini (Solaris & Husk)

When built against Houdini, CMake generates a Houdini package file at build/hdmitsuba.json. Copy it into your Houdini user preferences packages directory (for example ~/Library/Preferences/houdini/22.0/packages/ on macOS, or ~/houdini22.0/packages/ on Linux):

mkdir -p ~/Library/Preferences/houdini/22.0/packages
cp build/hdmitsuba.json ~/Library/Preferences/houdini/22.0/packages/

Once installed:

  • Solaris Viewport: Launch Houdini, switch to the Solaris (Stage) context, and select Mitsuba from the viewport renderer menu. Interactive settings are exposed in Display Options (D $\rightarrow$ Mitsuba), or can be authored as mitsuba:* attributes on a RenderSettings prim.
  • Husk CLI: Render USD stages from the command line via husk --renderer HdMitsubaRendererPlugin scene.usd -o output.exr.

Static Type Checking (Pyrefly)

We use pyrefly for static type checking of the Python code. A local pyrefly.toml configuration file is automatically generated by CMake in your project root.

To run the type checker, ensure your environment is set up and execute:

source build/setpath.sh
pyrefly check

Note: Currently Pyrefly is configured to not check USD or Mitsuba types. It's a useful sanity check, but not a guarantee of overall code correctness. The rigor of the type checking may be expanded in the future.

Python Code Formatting (Pyink)

Python code follows the Google Python style (2-space indentation, 80 columns). We use pyink to format the code and isort to sort imports. Both are configured in pyproject.toml, and CI checks that the code is formatted. Formatter output can change between releases, so install the versions pinned in the CI configuration:

pip install pyink==26.5.1 isort==9.0.2

# Format all Python files
pyink .
isort .

C++ Code Formatting (clang-format)

C++ code follows the Google C++ style and is formatted with clang-format, configured in .clang-format. CI checks that the code is formatted. As for Python, install the version pinned in the CI configuration:

pip install clang-format==23.1.5

# Format all C++ files
git ls-files '*.cc' '*.h' | xargs clang-format -i

Limitations and Future Work

While hdMitsuba already supports many Mitsuba & USD features, it currently has a number of known limitations. Some areas of future work are:

  • Flexible plugin discovery: The list of discoverable Mitsuba plugins is currently hardcoded in hdmitsuba/sdr_discovery.cc. Custom Mitsuba plugins will not be automatically discovered by USD applications without adding them to this list and rebuilding the delegate.
  • Custom Emitter Support: Currently, we only support standard USD emitters. In the near future, we will add support to specify custom Mitsuba textures on emitters (e.g., to set up spectral emission).
  • Incremental Material Updates: Materials are currently re-instantiated when modified. It would be useful to extend material support to also support efficient incremental updates.
  • Performance: There is room to further reduce overheads and improve performance. Note that hdMitsuba also inherits Mitsuba's performance characteristics (e.g., high JIT costs for scenes containing a very large number of materials).
  • Volume rendering: Volume rendering is not yet supported.
  • Windows support: Mitsuba 3 is Windows compatible, but we have not tested hdMitsuba on Windows. The Windows CI setup is brittle and slow, and we therefore have temporarily disabled it.

Contributing

We welcome community contributions and bug reports. To contribute code, please see the instructions in CONTRIBUTING.

Citation

This project was created by Delio Vicini, and significant improvements and fixes were contributed by Philippe Weier, Alyssa Cheng, Oscar Lin and Rus Maxham. We also acknowledge Matt Estela, Luca Prasso, Iker J. de los Mozos, Stephan Garbin, Bernd Bickel, Vincent Huang, and Thabo Beeler for helpful discussions and support. If you use hdMitsuba in your research, please consider citing it using the following BibTeX entry:

@software{hdmitsuba,
	title   = {hdMitsuba: A Hydra Render Delegate for Mitsuba 3},
	author  = {Delio Vicini, Philippe Weier, Alyssa Cheng, Oscar Lin, Rus Maxham},
	year    = 2026,
	note    = {https://github.com/google/hdmitsuba},
	version = {0.1.0}
}

Disclaimer

This is not an officially supported Google product. This project is not eligible for the Google Open Source Software Vulnerability Rewards Program

About

USD Hydra delegate for Mitsuba 3

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

90 stars

Watchers

4 watching

Forks

Releases

Packages

Used by

Contributors

Languages