Repository navigation
fix(instrumentor): skip meta-var injection in Triton-decorated functions - #9
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
auto_meta_var_injector.pyinsertsupdate_meta(...)after every assignment to aDEFAULT_META_VARSname in every function of an instrumented file, including Triton kernels.Triton cannot compile calls to arbitrary Python functions, so a kernel line such as vLLM's
batch_idx = tl.program_id(0)fails withRuntimeError: Unsupported function referenced: <function update_meta ...>, and the instrumentedprogram cannot run even in eager mode.
trace_robust.pyalways passesmeta_var_names=None,so this cannot be turned off from the official flow.
Fix
visit_FunctionDefreturnsFalse,so the body and nested functions are skipped.
any dotted segment equals
triton. This covers@triton.jit(...),autotune,heuristics,stacked decorators,
from triton import jit as tjit,import triton as tr,from triton import *, and re-exports such as vLLM'sfrom vllm.triton_utils import triton.source must always be instrumented the same way.
kernel breaks compilation.
instrument_file_auto_meta_varsreports the number of skipped functions in a newskipped_functionsfield; existing fields are unchanged.Testing
tests/test_auto_meta_var_injector.py(29 tests): 23 fail on 339e225 and all pass with thischange, including a GPU smoke test that compiles and launches a real kernel
(RTX 3090, torch 2.13, triton 3.7.1).
all 2879 vLLM files plus OpGuard's own
examples/andtests/. The only differences are insideTriton-decorated functions (46 → 0 injections there); injections outside them are unchanged
(116 → 116 in vLLM, per file and in total), and files whose only injections were in kernels are
now left byte-identical.
Out of scope
@cute.jit(CuTe DSL) functions in vLLM are still injected. This is the same class ofproblem; a follow-up can extend the rule to other JIT decorators (including
torch.jit.script).kernel = triton.jit(fn)is not detected.tests/test_instrumentor.py,tests/test_meta_vars.pyandtests/test_trace_with_meta.pyalready fail on main due to stale imports, independently of this change.