~ SyncSink.wasm - Synchronize media files by audio-to-audio alignment» By Joren on Tuesday 06 September 2022
I have built a tool for audio-to-audio alignment. It has applications for synchronization of media files. It works in the browser and you can synchronize your media files here with SyncSink.wasm. SyncSink.wasm does the following:
- From an incoming media-file audio is extracted, downmixed to mono and and resampled. This is done with ffmpeg.audio.wasm a wasm version of ffmpeg.
- For each audio track, fingerprints are extracted. These fingerprints reduce the the search space for alignment drastically.
- Each list of fingerprints is aligned with the list of fingerprints from the reference. Resulting in a rough alignment
- Cross correlation is done to refine the alignment resulting in sample accurate results.
Fig: media synchronization with audio-to-audio alignment.
It supports small time-scale adjustments of around 5%: audio alignment can still be found if audio speed differs a bit.
Some potential use cases where it might be of use:
- To stitch partially overlapping audio recordings together resulting in a single long audio recording.
- To synchronize multiple independent video recordings of the same event each with an audio recording of the environment.
- To align a high quality microphone recording with video/low-quality audio recording of the same event. The low quality audio recorded with a camera can then be replaced with the high quality microphone audio.
The code can be found in the SyncSink.wasm GitHub repository