Skip to content

oto: report when frames are played on Windows with WASAPI - #335

Open
marrasen wants to merge 2 commits into
ebitengine:mainfrom
marrasen:windows-played
Open

marrasen wants to merge 2 commits into
ebitengine:mainfrom
marrasen:windows-played

Conversation

@marrasen

Copy link
Copy Markdown
Contributor

This PR was written by Claude (Anthropic), an AI agent, at Marcus's request.

What issue is this addressing?

Updates #311

What type of issue is this addressing?

feature

What this PR does | solves

The WASAPI backend now reports when frames are played, with Mux.SetPlayed, so UnplayedSize counts the frames in the endpoint buffer and the delay Windows reports for the device. WinMM still reports nothing.

After each write, IAudioClock::GetPosition gives the stream's position at a performance-counter moment. The frame played then is the mix frame at the stream's position 0, plus the position converted to frames.

The position counts from the stream's start, and after Start it keeps counting at a uniform rate, also while the stream runs dry. So that position 0 stays a known frame of the mix:

  • The backend fills the buffer before Start.
  • While the context is suspended, the render thread stops the stream, and starts it again on Resume. Stop freezes the position.
  • When a write finds the buffer empty a whole buffer's time after the last fill, the stream ran dry. The backend then stops, resets, fills and starts it again, and clears the report.

Suspend clears the report. A report measured as Suspend runs is dropped.

One change in behaviour: before, the audio left in the buffer played out after Suspend. Now Suspend stops the stream within one device period, and that audio, at most one buffer (50 ms by default), plays after Resume.

Bluetooth

Windows doesn't count a Bluetooth headset's delay in the position, at least for these earbuds. On a Windows 11 laptop (10.0.26300), with the built-in speakers and Sony WF-1000XM4 earbuds:

Speakers WF-1000XM4
UnplayedSize − BufferedSize, playing 60–70 ms 60–70 ms
Taps on a click each second, offset from the click, with UnplayedSize +32 ms +263 ms
The same with BufferedSize +100 ms +330 ms

The tap test plays a click each second, and the listener presses Enter on each one, 20 times. Each press records how far past the nearest click the frame playing then is, as the bytes read less UnplayedSize (or less BufferedSize). The listener's own timing is the same on both devices, so the 231 ms between the two UnplayedSize medians is the earbuds' delay that Windows doesn't report. Once, after the earbuds reconnected, Windows reported about 110 ms for them, which is still short.

So on Bluetooth, UnplayedSize counts the part of the delay Windows knows about and not the rest.

Testing

CI can't play sound, so the checks above were run by hand, along with these:

  • Pause: the difference falls to 0 within 0.1 s.
  • Suspend: it is 0 throughout.
  • Resume: it is back within 0.1 s.
  • Five short suspend and resume cycles each report again.
  • With a click playing, the earbuds were put in their case and taken out, and the default output was switched twice in Sound settings. Each time, the clicks went on from the new device after a short silence, and the reports came back within 1.25 s, with no errors.

The tool used is on a branch of my fork, outside this PR: internal/wintest.

I could not test a stream running dry, as nothing here makes the render thread late on purpose.

🤖 Generated with Claude Code

Marcus Johansson and others added 2 commits November 14, 2026 09:24
After each write, the WASAPI driver reports the frame of the mix that
IAudioClock::GetPosition says is played, at the moment the position
was read.

The position counts from the stream's start, and keeps counting while
the stream runs dry. So the driver fills the buffer before Start, stops
the stream while the context is suspended, and when it finds the buffer
empty, it stops, resets and fills the stream again. Each time, the
stream's position 0 is a known frame of the mix.

WinMM still reports nothing.

Updates ebitengine#311

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clear the report when a dry stream restarts, as the frames it named
belong to the old stream. Restart only when a whole buffer's time has
passed since the last fill: an empty buffer sooner than that can be
the engine taking the last period, with no silence played. A clock
error other than a lost device now skips the report.

Updates ebitengine#311

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@hajimehoshi hajimehoshi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review written by Claude (Claude Code), on behalf of @hajimehoshi.

Thanks for the PR. The reporting itself looks right, but the central premise did not hold when tested, and most of the added complexity rests on it.

Main finding: the clock does not count while the stream is dry

The description says the IAudioClock position "keeps counting at a uniform rate, also while the stream runs dry", and notes this could not be tested. I tested it on a Windows 11 ARM64 VM (Parallels) by holding c.m from a test for 600 ms to starve the render thread, and sampling GetPosition and GetCurrentPadding (48000 Hz, stereo, default 2400-frame buffer):

State Position (frames) Frames written since position 0 Padding
Running 48213 51360 2400
Starved, 100 ms to 600 ms 51360 (frozen) 51360 0
Suspended (stream stopped) 16320 (frozen) 18720 2400

The position stopped exactly at the number of frames written and stayed there for the whole 500 ms, so it counts frames consumed from the client, not elapsed time.

If that holds on real hardware, three parts of this PR are unnecessary:

  • The dry restart (Stop/Reset/startFilled, filledAt, IAudioClient2::Reset): with the position frozen at the frames written, streamStart + pos is already correct after a dry period. In my run the restart did fire on recovery, costing an extra stop and start during a glitch.
  • Stopping the stream on Suspend: a suspended stream runs dry, the position freezes, and it continues on Resume. Dropping this also avoids the behaviour change where up to one buffer of old audio plays after Resume.
  • Pre-filling before Start: harmless, but no longer needed for position 0 to be a known frame.

Caveat: the device was Parallels' virtual audio, not a physical driver. Could you run the same starvation check on the laptop speakers and the earbuds? Taking c.m from a test in package oto for a few hundred milliseconds, then reading c.audioClock.GetPosition() against c.mixed - c.streamStart, is enough. If the position freezes there too, please simplify accordingly.

Other issues

  1. The Xbox E_OUTOFMEMORY retry regresses. startOnCOMThread now calls GetBuffer through startFilled. If it returns E_OUTOFMEMORY inside restart(), isWASAPIDeviceTransientError is false, so the context fails on the second occurrence instead of after wasapiOOMRetryLimit retries. At the first start it would fall back to WinMM for good.
  2. The restart heuristic can misfire. If the render thread wakes one full buffer after the last fill, just after the engine took the last period, paddingFrames == 0 and time.Since(c.filledAt) >= bufferTime both hold with no silence played. This goes away if the restart is dropped.
  3. queryPerformanceCounter100ns does avoidable work on every write. It calls QueryPerformanceFrequency through LazyProc.Call every device period. The frequency is fixed at boot and can be read once.
  4. Minor: wasapiContext.mixed is also written from restart() without c.m. That is safe today because the goroutines are serialized, but the field comment should say what guards it.

What looks good

  • The SetPlayed arithmetic, the QPC to time.Time conversion, and the S_FALSE handling in GetPosition and Reset are correct.
  • Lock ordering between c.m, suspendedCond.L and the mux is consistent, and a report racing with Suspend is dropped properly.
  • ClearPlayed on release, and degrading to no reports when the clock is unavailable, are right.
  • It builds for windows/amd64, 386 and arm64, and in my run reports resumed after starvation and after suspend/resume with no errors.

I did not run audible or Bluetooth checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants