Repository navigation
Keep damaged audio packets from stopping synchronization - #18
Merged
Merged
Conversation
Symphonia's AAC decoder reports a truncated or corrupted packet as an I/O
error ("unexpected end of bitstream"). Align treated it as fatal, so one
damaged packet in one camera file stopped the whole run without naming the
file (#16). A rejected packet now becomes silence of its recorded duration,
so the rest of the file is still used and keeps its timing.
Symphonia is chosen only when the first decodable packet does not yield far
fewer samples than its container records; HE-AAC, whose SBR layer Symphonia
lacks, yields half and goes to FFmpeg. M4A is decoded like MP4, with its
edit list. When the in-process decoder (Symphonia, or AVFoundation on
macOS) still fails on a file during analysis, FFmpeg decodes it again from
the start, also for a mix of all streams. Only a file that neither can read
is reported, in a warning that names it, and the other files still
synchronize. When only some streams of a file can be read, refinement uses
the stream analysis kept. Cancellation ends the run, and so does a missing
FFmpeg except on macOS, where Align does not ship it.
Symphonia 0.5 ignores the probe hint and scans the first megabyte of any
file for MP3 or ADTS sync words. In MXF video it finds them and decodes
video data as audio, and when the reader rejects the match it can read the
whole file. Containers are now opened by the reader their extension names;
MXF, MTS and R3D go straight to FFmpeg.
Fingerprints cached by the previous decoder are not reused (portable4).
The status switches to Analyzing when audio analysis starts, instead of
showing Reading metadata until the first file finishes, and analysis
progress counts are published in order.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #16.
On Windows and Linux, Align decodes MP4, MOV and WAV audio with Symphonia.
Its AAC decoder reports a truncated or corrupted packet as an I/O error,
"unexpected end of bitstream". Align treated every such error as fatal, and
one file's failure stopped the whole run, without naming the file.
Reproduced with a camera-original Canon MP4 (
CanonMP4_002, AAC-LC) with oneaudio packet cut short: 0.3.2 stops with exactly the reported
Decode error: symphonia: unexpected end of bitstream., and this branchsynchronizes the file at the same offset as the intact copy.
What changes for the user
Nothing is dropped that a decoder can read:
the file is used and keeps its timing, which FFmpeg itself does not do: it
drops such packets, and sometimes stops the stream.
fewer samples (under three quarters) than its container records. HE-AAC,
whose SBR layer Symphonia lacks, yields half and did not synchronize; it now
goes to FFmpeg. PCM is not checked, and short final packets pass. M4A is
decoded like MP4, with its edit list.
fails on a file during analysis, FFmpeg decodes it again from the start,
into fresh state, also for a mix of all streams.
and the other files still synchronize. In 0.3.2 such a file stopped the
whole run. When only some streams of a file can be read, refinement uses the
stream the analysis kept.
Align does not ship FFmpeg and AVFoundation's verdict on a file stands.
portable4, sofingerprints the previous decoder produced (HE-AAC, misdetected MXF) are not
reused. The first synchronization after the update analyses files again.
Symphonia 0.5 also ignores the probe hint and scans the first megabyte of any
file for MP3 or ADTS sync words. It detected a camera-original Canon EOS C70
XF-AVC MXF as 32 kHz MPEG audio after reading 4.3 MB, decoded generated MXF
files as 11 kHz MP3 noise, and read whole files (160 MB of 160 MB) when the
reader rejected its match. Containers are now opened by the reader their
extension names: MXF, MTS and R3D go straight to FFmpeg.
The status switches to "Analyzing" when audio analysis starts. Before, it
showed "Reading metadata" until the first file was analysed, which is where
the report saw the error. Analysis progress is counted and published under
one lock, so the completed count no longer goes backwards when workers
finish together.
Decoder differential
59 files carrying the same noise at a known offset: WAV (8 to 64 bit, RF64,
6 channels), AIFF, MP3, AAC-LC/HE-AAC/HE-AACv2/ALAC in M4A, MOV with PCM, AAC
and ALAC, MP4 (plain, faststart, fragmented), MXF, MTS with AC-3, AAC and
LPCM, files with damaged packets, a truncated file, a zeroed region, files
with no audio, plus camera-original Canon C70 MXF, Canon MP4 and Canon 6D
MOV. Each was synchronized against a reference on the portable backend, and
FFmpeg decoded each one independently.
except these, which improve:
The differential also shows offsets that are wrong in 0.3.2 and unchanged
here, for separate follow-up: MP3 by 23 to 25 ms (encoder delay not
trimmed), HE-AAC by 64 ms (also through FFmpeg), fragmented MP4 and MTS with
AAC by 21 ms (AAC priming), MTS with AC-3 by 5 ms.
Tests
Each fails on
mainor on the previous revision of this branch for theintended reason:
damaged_packet_becomes_silence_of_its_durationstreams_decoded_shorter_than_their_packets_go_to_ffmpegmp3_frames_inside_other_containers_are_not_decoded_as_audioaac_priming_matches_ffmpeg_timingnow also covers M4Ashort_final_pcm_packets_still_decode(the last milliseconds of an AIFF)in_process_decode_failures_are_retried_with_ffmpeg, for one stream and forall streams mixed; only one of the two files fails, so samples kept from the
failed attempt would shift it
refinement_uses_the_stream_that_could_be_readanalysis_progress_never_goes_backwards(deterministic: the listener holdsodd counts, so overlapping publication would reorder them)
unreadable_audio_is_reported_and_the_rest_still_synchronizes(the audiodata is missing, so neither decoder can read it)
The retry and stream-selection tests were also checked by mutation: keeping
the failed attempt's samples, skipping the retry for mixed streams, or the
old stream selection each makes them fail.
Validation
cargo fmt --all -- --checkcargo clippy --locked --workspace --all-targets -- -D warningscargo test --locked --workspace: 381 passed, 1 ignoredreleaseandpreserve-editingpass. The AAF gates were not run (no AAF module orpyaaf2 on this machine); this change does not touch AAF.
The reporter's files are not available. A damaged packet reproduces the
exact message; the MXF misdetection is a separate defect found on footage
from the same camera family.