~ SyncSink.wasm - Synchronize media files by audio-to-audio alignment

I have built a tool for audio-to-audio alignment. It has applications for synchronization of media files. It works in the browser and you can synchronize your media files here with SyncSink.wasm. SyncSink.wasm does the following:

1. From an incoming media-file audio is extracted, downmixed to mono and and resampled. This is done with ffmpeg.audio.wasm a wasm version of ffmpeg.
2. For each audio track, fingerprints are extracted. These fingerprints reduce the the search space for alignment drastically.
3. Each list of fingerprints is aligned with the list of fingerprints from the reference. Resulting in a rough alignment
4. Cross correlation is done to refine the alignment resulting in sample accurate results.

It supports small time-scale adjustments of around 5%: audio alignment can still be found if audio speed differs a bit.

Some potential use cases where it might be of use:

• To stitch partially overlapping audio recordings together resulting in a single long audio recording.
• To synchronize multiple independent video recordings of the same event each with an audio recording of the environment.
• To align a high quality microphone recording with video/low-quality audio recording of the same event. The low quality audio recorded with a camera can then be replaced with the high quality microphone audio.

The code can be found in the SyncSink.wasm GitHub repository

Code, ISMIR, and UGent

media_sync_recording.apng