Repository navigation
Startup: build the audio cache in the background and cut its build time from ~3 min to ~25 s - #397
Open
robert-d-schultz wants to merge 8 commits into
Open
robert-d-schultz wants to merge 8 commits into
robert-d-schultz wants to merge 8 commits into
Conversation
SearchFiles matches extensions as substrings, so the ".dat" lookups in the audio cache also returned the 256 terrain hf_height_map.data files in WH3. Each was decompressed and parsed as a sound dat, failed, and the failure was discarded. They also went into the audio fingerprint, and a terrain-only mod counted as having audio files. DatLoader also re-resolved every file's path through GetFullPath, which on the cached game container is a full table scan per file (~0.2s x 270). It now filters on the paths SearchFiles already returned, and logs dat files that fail to parse instead of dropping the list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Inserting WH3's 3.57M HIRC rows while SQLite keeps three secondary indexes up to date (one on the effectively random HircId) is the bulk of the bank-indexing phase. Replaying the real rows: 59.5s with indexes in place, 10.7s when they are created after the insert. Save now drops the Hircs/Didx indexes EF created, inserts, then recreates them from their original SQL inside the same transaction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
App.OnStartup built the game files audio cache on the UI thread before the main window was shown (added with the audio data cache, PR donkeyProgramming#386). After a game update that is minutes of a blank screen, for every user, whether or not they use the audio tools. It now runs on a background task. AudioRepository already builds a missing cache on first use; cache load/build now goes through a lock, so an audio tool opened while the prebuild is still running waits for it instead of recreating the same file (which the new concurrency test shows collides). The prebuild also resolved the scoped IAudioRepository from the root provider, which throws when scope validation is on (debugger attached), so it never ran there. It now creates its own scope. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cache file name contains a fingerprint of the game's packs, so every game update writes a new ~300MB Warhammer III cache and leaves the old one behind. The audio cache already cleans up after itself; the pack cache now does the same, removing only files named like this cache with a different 64-hex fingerprint. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Failed to load from cache - Generating new cache" is shown on every first run and after every game update, where nothing has failed. Say what is happening and why instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Six of the seven "known broken" animations are 0 byte entries in WH3's
anim2.pack. WH3 has 15 such files, so the other nine each logged an Error
with a stack trace ("File empty") on every startup, while the listed ones
logged a Warning each.
Empty files are now skipped generically and reported in one Information
line per container. The list keeps only the pre-WH3 roman_aqueduct entry,
which is not in WH3's packs and so could not be checked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The startup audio cache build now logs from a background thread while the UI thread logs too; the history queue was unsynchronised. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DerpyNewb
added a commit
to DerpyNewb/TheAssetEditorDN
that referenced
this pull request
Nov 19, 2026
… audio cache in the background Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> # Conflicts: # AssetEditor/App.xaml.cs
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
After a game update, Asset Editor can take minutes to open. A user reported 5 minutes.
App.OnStartupbuilds the game files audio cache on the UI thread before showing the main window (added with the audio data cache in #386), so for every user, whether or not they use the audio tools, the screen stays blank until it finishes. On my WH3 install the build took about 3 minutes:SearchFilesmatches extensions as substrings, so the.datlookup also returns the 256 terrainhf_height_map.datafiles (31 MB compressed). Each one was decompressed and parsed as a sound dat, failed, and the failure was thrown away.DatLoaderthen looked up every file's path again throughGetFullPath, which on the cached game container is a full table scan (~0.2 s × 270).HircId.Startup also logged 9
Errors with stack traces ("File empty") from the skeleton/animation lookup. They come from empty.animfiles that aren't on its hardcoded "known broken" list.Changes (one commit each)
.dat/.bnk/.wwiseidsfiles. It re-checks the extension afterSearchFilesand filters on the paths it already has instead of callingGetFullPath. Dat files that fail to parse are now logged instead of dropped. This also stops terrain files from changing the audio fingerprint, and stops a terrain-only mod from counting as having audio files.AudioRepositoryalready builds a missing cache on first use; loading or building now goes through a lock, so an audio tool opened while the prebuild runs waits for it instead of recreating the same file. The prebuild also used to resolve the scopedIAudioRepositoryfrom the root provider, which throws when scope validation is on (debugger attached), so it never ran there. It now creates its own scope..animfiles are skipped generically. Six of the seven "known broken" anims are 0-byte entries in WH3'sanim2.pack, and WH3 has 15 such files. All 15 are now skipped with one Information line. The list keeps only the pre-WH3roman_aqueductentry, which isn't in WH3's packs, so I couldn't check it.Not changed:
SearchFilesitselfThe root cause of (1) is that
SearchFilesmatches extensions withContains. I didn't change that, becauseSearchFiles_ExtensionFilter_MatchesFilenameSubstrings_ForLegacyWemVariantsasserts that behaviour (.wemmatching.wem_temp/.wem.{sdf}). Vanilla WH3 has no files like that, and no code searches for.wem. The pack browser has the same issue: filtering for.datlists the heightmaps, and filtering for.animlists.animpackfiles. Is the substring match still needed? If not, I can switch the containers to a real extension match in a follow-up.Verification
.datafiles don't affect the fingerprint; indexes exist afterSave; concurrentLoadOrBuildCachecallers all get a working cache (fails 6/6 without the lock); stale pack caches are removed and nothing else is. Each test fails without its fix.Test.Audio,Shared.CoreTestandGameWorld.CoreTestall pass.The new logging shows that 5 sound dats in vanilla WH3 already failed to parse before this change (
audio_rules_battle_order_vo,audio_rules_campaign_conversation_vo,audio_rules_campaign_order_vo,english(uk)\advisor\wwise.dat,tools_data.dat). They're probably other formats that belong on the skip list, but I haven't checked, so I left that list unchanged.🤖 Generated with Claude Code