Panako is an acoustic fingerprinting system. The system is able to extract fingerprints from an audio stream, and either store those fingerprints in a database, or find a match between the extracted fingerprints and stored fingerprints. Several acoustic fingerprinting algorithms are implemented within Panako. The main algorithm, the Panako algorithm, has the feature that audio queries can be identified reliably and quickly even if they has been sped up, time stretched or pitch shifted with respect to the reference audio. The main focus of Panako is to serve as a demonstration of the Panako algorithm. Other acoustic fingerprinting schemes are implemented to serve as a baseline for comparison. More information can be found in the article about Panako.
Please be aware of the patents US7627477 B2 and US6990453 and perhaps others. They describe techniques used in some algorithms implemented within Panako. The patents limit the use of some algorithms under various conditions and for several regions. Please make sure to consult your intellectual property rights specialist if you are in doubt about these restrictions. If these restrictions apply, respect the patent holders rights. The first aim of Panako is to serve as a research platform to experiment with and improve fingerprinting algorithms.
This document covers installation, usage and configuration of Panako.
The Panako source code is licensed under the GNU Affero General Public License.
Content based music search algorithms make it possible to identify a small audio fragment in a digital music archive with potentially millions of songs. Current search algorithms are able to respond quickly and reliably on an audio query, even if there is noise or other distortions present. During the last decades they have been used successfully as digital music archive management tools, music identification services for smartphones or for digital rights management.
Fig. General content based audio search scheme.
Most algorithms, as they are described in the literature, do not allow substantial changes in replay speed. The speed of the audio query needs to be the same as the reference audio for the current algorithms to work. This poses a problem, since changes in replay speed do occur commonly, they are either introduced by accident during an analog to digital conversion or are introduced deliberately.
Analogue physical media such as wax cylinders, wire recordings, magnetic tapes and grammophone records can be digitized at an incorrect or varying playback speed. Even when calibrated mechanical devices are used in a digitization process, the media could already have been recorded at an undesirable speed. To identify duplicates in a digitized archive, a music search algorithm should compensate for changes in replay speed
Next to accidental speed changes, deliberate speed manipulations are sometimes introduced during radio broadcasts: occasionally songs are played a bit faster to fit into a timeslot. During a DJ-set speed changes are almost always present. To correctly identify audio in these cases as well, a music search algorithm robust against pitch shifting, time stretching and speed changes is desired.
The Panako algorithm allows such changes while maintaining other desired features as scalability, robustness and reliability. Next to the panako algorithm there is also an implementation of the algorithm described in An Industrial-Strength Audio Search Algorithm (internally identified as Olaf). Also the algorithm in A Robust Audio Fingerprinter Based on Pitch Class Histograms – Applications for Ethnic Music Archives is available. To make comparisons between fingerprinting systems easy, researchers are kindly invited to contribute algorithms to the Panako project.
Alternative open source music identification systems are audfprint, NeuralFP and echoprint. Alternative systems with similar features are described in US7627477 B2 and in Quad-based Audio Fingerprinting Robust To Time And Frequency Scaling by Reinhard Sonnleitner and Gerhard Widmer.
Panako is organized as follows:
src
contains the Java source files.lib
contains the dependencies (JAR-files).build
contains an ant build file to compile Panako and the documentation. Use ant release
to build the source and compile Panako.scripts
contains helper scripts for evaluation and to analyse resultsTo compile Panako, the JDK 11 or later is required. See the installation instructions on the Java website, for installation on your operating system.
Panako uses the Gradle build system. Install it on your system.
Once Gradle and the other components are installed correctly the following commands should get you started:
git clone --depth 1 https://github.com/JorenSix/Panako.git
cd Panako
./gradlew shadowJar #Builds the core Panako library
./gradlew install #Installs Panako in the ~/.panako directory
./gradlew javadoc #Creates the documentation in doc
sudo cp resources/defaults/panako /usr/local/bin #copies the panako startup script to your path
The last command copies the startup script in resources/defaults/panako
to a directory in your path. The script allows for easy access to Panako’s functionality. If this does not work for you, you can still call Panako using java -jar ~/.panako/panako.jar [..args]
.
Panako decodes audio using by calling ffmpeg
, an utility for decoding and resampling audio.
The ffmpeg
command needs to on the systems path. On a Debian like system:
apt-get install ffmpeg
Test Panako. You might need a new shell to use panako
. Note that you also need SoX and wget on your system:
#brew install sox wget
#apt-get install sox wget
panako -v #prints version
panako stats #db info
cd resources/scripts/create_test_dataset
ruby create_dataset.rb #make sure sox
panako store dataset/*
panako query query/*
Panako provides a command line interface, it can be called using panako subapplication [argument...]
. For each task Panako has to perform, there is a subapplication. There are subapplications to store fingerprints, query audio fragments, monitor audio streams, and so on. Each subapplication has its own list of arguments, options, and output format that define the behavior of the subapplication. The screen capture below shows a typical interaction with Panako.
Fig. A typical interaction with Panako via the command line interface. It shows how to store audio, perform a query and how to print database statistics.
To save some keystrokes the name of the subapplication can be shortened using a unique prefix. For example panako m file.mp3
is expanded to panako monitor file.mp3
. Since both stats
and store
are valid subapplications the store
call can be shortened to panako sto *.mp3
, panako s *.mp3
gives an invalid application message. A trie is used to find a unique prefix.
What follows is a list of those subapplications, their arguments, and respective goal.
panako store
The store
instruction extracts fingerprints from audio tracks and stores those in the datastorage. The command expects a list of audio files, video files or a text file with a list of file names.
#Audio is converted automatically panako store audio.mp3 audio.ogg audio.flac audio.wav
#The first audio stream of video containers formats are used. panako store audio.mpc audio.mp4 audio.acc audio.avi audio.ogv audio.wma
#Glob characters can be practical panako store */*.mp3
#A text file with audio files can be used as well #The following searches for all mp3's (recursively) and #stores them in a text file find . -name '*.mp3' > list.txt #Iterate the list panako store list.txt
panako delete
This application removes fingerprints from the index. It essentially reverses the store
operation. The operation can be checked with panako stats
panako delete test.mp3
The default key-value-store is backed by some kind of B-tree structure. Removing many elements from such structure might leave the tree in an unbalanced state, which results in worse performance. I am not sure about the performance implications of deletion for LMDB but it might be of interest to either rebuild the index or avoid deletion as much as possible.
panako query
The query
command extracts fingerprints from an audio frament and tries to match the fingerprints with the database.
panako query short_audio.mp3
panako monitor
The query
command extracts fingerprints from a short audio frament and tries to match the fingerprints with the database. The incoming audio is, by default, chopped in parts of 25s with an overlap of 5s. So every 25-5=20s
the database is queried and a result is printed to the command line.
panako monitor [short_audio.mp3]
If no audio file is given, the default microphone is used as input.
panako stats
The stats
command prints statistics about the stored fingerprints and the number of audio fragments. If an integer argument is given it keeps printing the stats every x seconds.
panako stats # print stats once
panako stats 20 # print stats every 20s
panako config
The config
subapplication prints the configuration currently in use.
panako config
To override configuration values there are two options. The first option is to create a configuration file, by default at the following location: ~./panako/config.properties
. The configuration file is a properties text file. An commented configuration file should be included in the doc directory at doc/config.properties
.
The second option to override configuration values is by adding them to the arguments of the command line call as follows:
panako subapplication CONFIG_KEY=value
For example, if you do not want to check for duplicate files while building a fingerprint database the following can be done:
panako store file.mp3 CHECK_FOR_DUPLICATES=FALSE
The configuration values provided as a command line argument have priority over the ones in the configuration file. If there is no value configured a default is used automatically. To find out which configuration options are available and their respective functions, consult the documented example configuration file doc/config.properties
..
panako resolve
This application simply returns the identifier that is used internally for a filename. The following call returns for example 54657653
:
panako resolve test.mp3
The internal identifiers are currently defined using integers.
panako same
This application checks the similarity of two files. The percentage returned indicates the percentage of seconds for which fingerprints match between the first and second file. So 100% that matches are found in every second. A result of 30% still means that much of the audio matches.
panako same first.mp3 second.mp3
Note that this operation is performed in memory. Nothing changes on disk.
Panako can also be ran in a containerized environment. A Dockerfile
is provided which should both work on x86_64
as aarch64
. To build the container and run commands the following should get you started. Note that the database with fingerprints is located on the host at ~/.panako/docker_dbs
:
cd resources/scripts/
docker build -t panako:2.1 .
mkdir -p $HOME/.panako/docker_dbs
wget "https://filesamples.com/samples/audio/mp3/sample3.mp3"
docker run -v $HOME/.panako/docker_dbs:/root/.panako/dbs panako:2.1 panako store sample3.mp3
docker run -v $HOME/.panako/docker_dbs:/root/.panako/dbs panako:2.1 panako stats
Some relevant reading material about acoustic fingerprinting. The order gives an idea of relevance to the Panako project.
The Panako software was developed at IPEM, Ghent University by Joren Six.
Some parts of Panako were inspired by the Robust Landmark-Based Audio Fingerprinting Matlab implementation by Dan Ellis.
If you use Panako for research purposes, please cite the following works:
@inproceedings{six2014panako,
author = {Joren Six and Marc Leman},
title = {{Panako - A Scalable Acoustic Fingerprinting System Handling Time-Scale and Pitch Modification}},
booktitle = {{Proceedings of the 15th ISMIR Conference (ISMIR 2014)}},
year = 2014
}
@inproceedings{six2021panako,
title={Panako 2.0-Updates for an acoustic fingerprinting system},
author={Six, Joren},
booktitle={Demo / late-breaking abstracts of 22st International Society for Music Information Retrieval Conference, ISMIR},
year={2021}
}
The directory scripts/reproducibility
contains scripts to reproduce the result found in the
Panako ISMIR 2014 paper. The scripts follow this procedure:
To run the evaluation scripts, a UNIX like system with following utilities is required:
Evidently the panako software should also be installed. Please see above to install the Panako.
If all requirements are met, running the test is done using bash run_experiment.bash
Improvements to SyncSink. The cross correlation now behaves well also in edge cases. The Sync code has been simplified to allow maintenance. Included unit tests. Updated to TarsosDSP version 2.4.
This release adds a simplified version of chromaprint and an implementation of ‘A Highly Robust Audio Fingerprinting System’ by Haitsma and Kalker
A major overhaul of Panako. The main aim of this release is to ensure the longevity and maintainability of Panako. The featureset has been reduced to focus on core functionality. The version bump is also explained by the use of lambdas and the need for a newer JRE (8+). A list of changes:
scripts
directory. They help with evaluation, parsing results, checking results, building panako, creating documentation,…Changed the build system to Gradle, mainly to make upgrading dependencies more straightforward. The folder structure has been changed accordingly. The default JDK target has been changed to Java SE 11 (LTS). Panako now also supports Apple M1 targets.