~ Reproduction of speech using MIDI

Tarsos is now capable of reproducing speech using MIDI. The idea to convert speech into MIDI comes from the blog of Corban Brook where the following video can be found, actually a work by Peter Ablinger:

Another example of music inspired by speech is this interview with Louis Van Gaal:

Tarsos sends out midi data based on an FFT analysis of the signal. It maps the spectrogram to MIDI Messages and uses the power spectrum to calculate the velocity of each note on message.

The implementation can run in real-time but the output has some delay: the FFT calculation, constructing MIDI messages, calculating velocity, synthesizing sound, … is not instantaneous.

To use this capability Tarsos supports the following syntax. If a MIDI file is given the MIDI messages are written to the file. If an audio file is given Tarsos uses the audio as input. If the --pitch switch is used only the F0 is considered to construct MIDI messages instead of a complete FFT.

  


  
java -jar tarsos.jar pitch_to_midi [--pitch] [midi_out.midi] [audio_in.wav]

Java and HoGent

ttm.mp3, ttm.midi, sitting_in_a_room.mp3, sitting_in_a_room.midi, and tarsos.jar