Hi, I'm Joren. Welcome to my website. I'm a research software engineer in the field of Music Informatics and Digital Humanities. Here you can find a record of my research and projects I have been working on. Learn more »
This afternoon I created a screencast showing the main features of Tarsos. If everything goes well it should be visible below.
To give Tarsos a try you can start Tarsos using JAVA WebStart or download the executable Tarsos JAR-file. A JAVA 1.5 runtime is required.
HoGent, Computational ethnomusicology, and Tarsos
Yesterday Tarsos was publicly presented at the symposium Perspectives for Computational Musicology in Amsterdam. The first public presentation of Tarsos, excluding this website. The symposium was organized by the Meertens Institute on the occasion of Peter van Kranenburg’s PhD defense.
The presentation included a live demo of a daily build of Tarsos (a Friday evening build) which worked, surprisingly, without hiccups. The presentation was done by Olmo Cornelis. This was the small introduction:
Tarsos – a Platform for Pitch Analysis of Ethnic Music
Ethnic music is a vulnerable cultural heritage that has received only recently more attention within the Music Information Retrieval community. However, access to ethnic music remains problematic, as this music does not always correspond to the Western concepts of music and metadata that underlie the currently available content-based methods. During this lecture, we like to present our current research on pitch analysis of African music. TARSOS, a platform for analysis, will be presented as a powerful tool that can describe and compare scales with great detail.
To give Tarsos a try ou can start Tarsos using JAVA WebStart or download the executable Tarsos JAR-file. A JAVA 1.5 runtime is required.
Presentation
Tarsos now has an easy to use drag and drop User Interface. It can be used to extract tone scale information from audio files.
Start tarsos using JAVA WebStart.
Drag and drop works for scala tone scale files and different kinds of audio files. Audiofiles are transcoded automagically using an embedded ffmpeg binary which is platform dependend. It works on linux and windows, on other platforms only WAV files are supported.
Some of the current features:
HoGent, Java, featured, Tarsos, and Computational ethnomusicology
Tarsos can be used to render MIDI files to audio (WAV) files using arbitrary tone scales. This functionallity can be used to (automatically) verify tone scale extraction from audio files. Since I could not find a dataset with audio and corresponding tone scales creating one using MIDI seemed a good idea.
MIDI files can be found in spades, tone scales on the other hand are harder to find. Luckily there is one massive source, the Scala Tone Scale Archive: A large collection of over 3700 tone scales.
Using Scala tone scale files and a midi files a Tone Scale – Audio dataset can be generated. The quality of the audio depends on the (software) synthesizer and the SoundFont used. Tarsos currently uses the Gervill synthesizer. Gervill is a pure Java software synthesizer with support for 24bit SoundFonts and the MIDI tuning standard.
A recent version of the JRE needs to be installed on your system if you want to use Tarsos. Tarsos itself can be downloaded in the form of the Tarsos JAR Package.
Currently Tarsos has a Command Line Interface. An example with the files you can find attached:
1
java -jar tarsos.jar --midi BWV_1007.mid --scala 120.scl --out bach.wav
The result of this command should yield an audio file that sounds like the cello suites of bach in a nonsensical tone scale with steps of 120 cents. Executing tone scale extraction on the generated audo yields the expected result. In the pich class histogram every 120 cents a peak can be found.
To summarize: by rendering audio with MIDI and Scala tone scale files a dataset with tone scale – audio information can be generated and tone scale extraction algorithms can be tested on the fly.
This method also has some limitations. Because audio is rendered there is no (background) noise, no fluctuations in pitch and timbre,… all of which are present in recorded audio. So testing testing tone scale extraction algorithms on recorded audio remains advised.
Tarsos, featured, Music Information Retrieval, Computational ethnomusicology, and HoGent
tarsos.jar, 120.scl.txt, bach_BWV_1007_120.mp3, and BWV_1007.mid
Tarsos is now capable of reproducing speech using MIDI. The idea to convert speech into MIDI comes from the blog of Corban Brook where the following video can be found, actually a work by Peter Ablinger:
Another example of music inspired by speech is this interview with Louis Van Gaal:
Tarsos sends out midi data based on an FFT analysis of the signal. It maps the spectrogram to MIDI Messages and uses the power spectrum to calculate the velocity of each note on message.
The implementation can run in real-time but the output has some delay: the FFT calculation, constructing MIDI messages, calculating velocity, synthesizing sound, … is not instantaneous.
To use this capability Tarsos supports the following syntax. If a MIDI file is given the MIDI messages are written to the file. If an audio file is given Tarsos uses the audio as input. If the --pitch switch is used only the F0 is considered to construct MIDI messages instead of a complete FFT.
1
java -jar tarsos.jar pitch_to_midi [--pitch] [midi_out.midi] [audio_in.wav]
Tarsos can be used to search for music that uses a certain tone scale or tone interval(s). Tone scales can be defined by a Scala tone scale file or an exemplifying audio file. This text explains how you can use Tarsos for this task.
Scala files are text files with information about a tone scale. It is used to share and exchange tone scales. The file format originates from the Scala program :
Scala is a powerful software tool for experimentation with musical tunings, such as just intonation scales, equal and historical temperaments, microtonal and macrotonal scales, and non-Western scales. It supports scale creation, editing, comparison, analysis, …
The Scala file format is popular because there is a library with more than 3000 tone scales available on the Scala website.
Tarsos also understands Scala files. It is able to create a pitch class histogram using a gaussian mixture model. A technique described in A. C. Gedik, B.Bozkurt, 2010, "Pitch Frequency Histogram Based Music Information Retrieval for Turkish Music ", Signal Processing, vol.10, pp.1049-1063. (doi:10.106/j.sigpro.2009.06.017).
An example should make things clear. Lets search for an interval of 300 cents or exactly three semitones. A scala file with this interval is easy to define:
1
2
3
4
5
6
7
! example.scl
! An example of a tone interval of 300 cents
Tone interval of 300 cents
2
!
900
1200.0
The next step is to create a histogram with an interval of 300 cents. In the block diagram this step is called “Peak histogram creation”. The Similarity calculation step expects a list of histograms to compare with the newly defined histogram. Feeding the similarity calculation with the western12ET tone scale and a pentatonic Indonesian Slendro tone scale shows that a 300 cents interval is used in the western tone scale but is not available in the Slendro tone scale.
This example only uses scala files, creating histograms is actually not needed: calculating intervals can be done using the scala file itself. This changes when audio files are compared with each other or with scala files.
When audio files are fed to the algorithm additional steps need to be taken.
The last two steps are the same for audio files or scala files.
Using real audio files can cause dirty histograms. Determining how many distinct pitch classes are used is no trivial task, even for an expert (human) listener. Tarsos should provide a semi-automatic way of peak extraction: a best guess by an algorithm that can easily be corrected by a user. For the moment Tarsos does not allow manual intervention.
To use tarsos you need a recent java runtime (1.6) and the following command line arguments:
1
2
java -jar tarsos.jar rank --detector TARSOS_MPM
--needle audio.wav --haystack scala.scl other_audio.wav other_scala_file.scl
Tarsos, featured, Music Information Retrieval, Computational ethnomusicology, and HoGent
data_flow_audio.png, tarsos.jar, and data_flow_scala.png
This post is about the tools I use to keep the source code of Tarsos reasonably clean, consistent and readable. Static code analysis can be of great help if you want to maintain strict coding standards and follow language idioms. Some of the patterns they can detect for you:
And even more subtle, but equally important:
In a previous life I used .NET and the static code analysis tools FxCop & StyleCop. FxCop operates on bytecode (or intermediate language in .NET parlance) level, StyleCop analyses the source code itself. Tarsos uses JAVA so I looked for JAVA alternatives and found a few.
On freesoftwaremagazine.com there is an article series on JAVA static code analysis software. It covers PMD and FixBugs and integration in Eclipse. It does not cover Checkstyle. Checkstyle is essentialy the same as PMD but it is better integrated in eclipse: it checks code on save and uses the standard ‘Problems’ interface, PMD does not.
To fix problems Eclipse save actions can save you some time. IBM has an article on how to keep your code clean using Eclipse.
Continuous testing is also a really nice thing to have: detecting unexpected behavior while refactoring/programming can prevent unnecessary bug hunts. A video about immediate feedback using continuous testing makes this clear.
Another tip is a more philosophical one: making your code and code revisions publicly available makes you think twice before implementing (and subsequently publishing) a quick and dirty hack. Tarsos is available on github.
featured, HoGent, Java, and Code
I just finished creating a first release of Tarsos. The release contains several demo applications, some more usefull than other. Tarsos is a work in progress: not all functionality is exposed with the CLI demo applications. The demos should however give a taste of the possibilities. All demo applications follow this pattern:
1
java -jar tarsos.jar subcommand [--option [argument] ...]
To get help the --help switch can be used. It generates contextual help for either the subcommand or for Tarsos itself.
1
2
java -jar tarsos.jar --help
java -jar tarsos.jar subcommand --help
1
java -jar tarsos.jar detect_pitch --in flute.novib.mf.C5B5.wav
1
java -jar tarsos.jar midi_to_wav --midi satie_gymno1.mid --scala 120.scl
1
java -jar tarsos.jar audio_to_scala --in out.wav
1
java -jar tarsos.jar annotate --in out.wav
1
java -jar tarsos.jar pitch_table
Today I created a spectrogram application using Tarsos. The application listens to an audio input, computes an FFT and at the same time calculates pitch. The expected pitch is overlaid on the spectrogram. All this happens real-time and is implemented using JAVA.
This is the most recent version of the spectrogram implementation in java.
1
2
3
4
5
6
7
8
9
10
float pitch = Yin.processBuffer(buffer, (float) sampleRate);
fft.transform(buffer);
double maxAmplitude = 0;
for (int j = 0; j < buffer.length / 2; j++) {
double amplitude = buffer[j] * buffer[j] + buffer[j +
buffer.length/2] * buffer[j+ buffer.length/2];
amplitude = Math.pow(amplitude, 0.5);
colorIndexes[j] = amplitude;
maxAmplitude = Math.max(amplitude, maxAmplitude);
}
If you want to test it yourself download the spectrogram jar package and execute:
1
java -jar spectrogram.jar
The JAVA software program we are developing is called Tarsos and can now be found on GitHub. GitHub is a web-based hosting service for projects that use the Git version control system.
Currently Tarsos is a collection of Java classes to create, compare and process pitch-frequency data using histograms. In it’s current state it is not usable for end-users.
Tarsos is developed at University College Ghent, Faculty of Music and uses a number of open source libraries:
featured, Code, Java, HoGent, Computational ethnomusicology, and Tarsos