With the goal in mind to reduce common runner injuries we first need to measure some running style characteristics. Therefore, we have developed a sensor to measure how hard a runners foot repeatedly hits the ground. This sensor has been compared with laboratory equipment which proofs that its measurements are valid and can be repeated. The main advantages of our sensor is that it can be used 'in the wild', outside the lab on the runners regular tours. We want to use this sensor to provide real-time biofeedback in order to change running style and ultimately reduce injury risk.
We have published an article on this sensor in the journal of Biomechanics: Pieter Van den Berghe, Joren Six, Joeri Gerlo, Marc Leman, Dirk De Clercq [*Validity and reliability of peak tibial accelerations as real-time measure of impact loading during over-ground rearfoot running at different speeds*](https://www.sciencedirect.com/science/article/pii/S0021929019300764), ([author version](https://0110.be/files/attachments/466/author_version.pdf), Journal of Biomechanics, 2019
> Studies seeking to determine the effects of gait retraining through biofeedback on peak tibial acceleration (PTA) assume that this biometric trait is a valid measure of impact loading that is reliable both within and between sessions. However, reliability and validity data were lacking for axial and resultant PTAs along the speed range of over-ground endurance running. A wearable system was developed to continuously measure 3D tibial accelerations and to detect PTAs in real-time. Thirteen rearfoot runners ran at 2.55, 3.20 and 5.10 m\*s-1 over an instrumented runway in two sessions with re-attachment of the system. Intraclass correlation coefficients (ICCs) were used to determine within-session reliability. Repeatability was evaluated by paired T-tests and ICCs. Concerning validity, axial and resultant PTAs were correlated to the peak vertical impact loading rate (LR) of the ground reaction force. Additionally, speed should affect impact loading magnitude. Hence, magnitudes were compared across speeds by RM-ANOVA. Within a session, ICCs were over 0.90 and reasonable for clinical measurements. Between sessions, the magnitudes remained statistically similar with ICCs ranging from 0.50 to 0.59 for axial PTA and from 0.53 to 0.81 for resultant PTA. Peak accelerations of the lower leg segment correlated to LR with larger coefficients for axial PTA (r range: 0.64--0.84) than for the resultant PTA per speed condition. The magnitude of each impact measure increased with speed. These data suggest that PTAs registered per stand-alone system can be useful during level, over-ground rearfoot running to evaluate impact loading in the time domain when force platforms are unavailable in studies with repeated measurements.
---
## [ISMIR 2018 Conference - Automatic Analysis Of Global Music Recordings suggests Scale Tuning Universals](https://0110.be/posts/ISMIR_2018_Conference_-_Automatic_Analysis_Of_Global_Music_Recordings__suggests_Scale_Tuning_Universals.md)
- Published: 2018-09-24T00:00:00Z
- Updated: 2019-07-03T13:46:15Z
- Author: Joren
- ID: 462
- Canonical: https://0110.be/posts/ISMIR_2018_Conference_-_Automatic_Analysis_Of_Global_Music_Recordings__suggests_Scale_Tuning_Universals
- Tags: [IPEM](https://0110.be/tags/IPEM.md), [ISMIR](https://0110.be/tags/ISMIR.md), [Music Information Retrieval](https://0110.be/tags/Music%20Information%20Retrieval.md), [Presentation](https://0110.be/tags/Presentation.md), [Research papers](https://0110.be/tags/Research%20papers.md), [UGent](https://0110.be/tags/UGent.md)
Thanks to the support of a travel grant by the faculty of Arts and Philosophy of Ghent University I was able to attend the ISMIR 2018 conference. A conference on Music Information Retrieval. I am co author on a contribution for the [the Late-Breaking / Demos session](http://ismir2018.ircam.fr/pages/events-lbd.html)
> The structure of musical scales has been proposed to reflect universal acoustic principles based on simple integer ratios. However, some studying tuning in small samples of non-Western cultures have argued that such ratios are not universal but specific to Western music. To address this debate, we applied an algorithm that could automatically analyze and cross-culturally compare scale tunings to a global sample of 50 music recordings, including both instrumental and vocal pieces. Although we found great cross-cultural diversity in most scale degrees, these preliminary results also suggest a strong tendency to include the simplest possible integer ratio within the octave (perfect fifth, 3:2 ratio, \~700 cents) in both Western and non-Western cultures. This suggests that cultural diversity in musical scales is not without limit, but is constrained by universal psycho-acoustic principles that may shed light on the evolution of human music.
- [musical\_scales\_extended\_abstract\_\_0826\_revision\_copy.pdf](https://0110.be/files/attachments/462/musical_scales_extended_abstract__0826_revision_copy.pdf)
- [animated\_logo.gif](https://0110.be/files/attachments/462/animated_logo.gif)
- [ismir\_2018\_logo.jpg](https://0110.be/files/attachments/462/ismir_2018_logo.jpg)
---
## [JGaborator - Fast Gabor spectral transforms in Java](https://0110.be/posts/JGaborator_-_Fast_Gabor_spectral_transforms_in_Java.md)
- Published: 2018-09-13T00:00:00Z
- Updated: 2018-09-26T13:25:04Z
- Author: Joren
- ID: 465
- Canonical: https://0110.be/posts/JGaborator_-_Fast_Gabor_spectral_transforms_in_Java
- Tags: [Code](https://0110.be/tags/Code.md), [UGent](https://0110.be/tags/UGent.md)
Recently I have published a small library [on github called JGaborator](https://github.com/JorenSix/JGaborator). The library calculates fine grained constant-Q spectral representations of audio signals quickly from Java. The calculation of a [Gabor transform](https://en.wikipedia.org/wiki/Gabor_transform) is done by a C library named [Gaborator](http://gaborator.com). A Java native interface (JNI) bridge to the C Gaborator is provided. A combination of Gaborator and a fast FFT library (such as [pfft](https://bitbucket.org/jpommier/pffft)) allows fine grained constant-Q transforms at a rate of about 200 times real-time on moderate hardware. It can serve as a front-end for several audio processing or MIR applications.
For more information on the Gaborator C library by Andreas Gustafsson, please see the [gaborator.com](http://gaborator.com) website or a talk by the author on the library called [Exploring time-frequency space with the Gaborator](https://www.youtube.com/watch?v=ONJVJBmFiuE)
While the gaborator allows reversible transforms, only a forward transform (from time domain to the spectral domain) is currently supported from Java.A spectral visualization tool for sprectral information is part of this package. See below for a screenshot:
As an extension of the ISMIR conferences the International Society for Music Information Retrievel started a new journal: TISMIR. The first issue contains an article of mine:\
[A Case for Reproducibility in MIR: Replication of 'A Highly Robust Audio Fingerprinting System'](https://transactions.ismir.net/articles/10.5334/tismir.4/). The abstract can be read here:
> Claims made in many Music Information Retrieval (MIR) publications are hard to verify due to the fact that (i) often only a textual description is made available and code remains unpublished -- leaving many implementation issues uncovered; (ii) copyrights on music limit the sharing of datasets; and (iii) incentives to put effort into reproducible research -- publishing and documenting code and specifics on data -- is lacking. In this article the problems around reproducibility are illustrated by replicating an MIR work. The system and evaluation described in 'A Highly Robust Audio Fingerprinting System' is replicated as closely as possible. The replication is done with several goals in mind: to describe difficulties in replicating the work and subsequently reflect on guidelines around reproducible research. Added contributions are the verification of the reported work, a publicly available implementation and an evaluation method that is reproducible.
---
## [JNMR article - Beyond documentation – The digital philology of interaction heritage](https://0110.be/posts/JNMR_article_-_Beyond_documentation_%E2%80%93_The_digital_philology_of_interaction_heritage.md)
- Published: 2018-07-31T00:00:00Z
- Updated: 2018-09-26T13:13:46Z
- Author: Joren
- ID: 464
- Canonical: https://0110.be/posts/JNMR_article_-_Beyond_documentation_%E2%80%93_The_digital_philology_of_interaction_heritage
- Tags: [JNMR](https://0110.be/tags/JNMR.md), [Presentation](https://0110.be/tags/Presentation.md), [Research papers](https://0110.be/tags/Research%20papers.md), [UGent](https://0110.be/tags/UGent.md)
Marc Leman and myself have recently published an article in the Journal of New Music Research for a special issue on Digital Philology for Multimedia Cultural Heritage. Our contribution is titled 
[Beyond documentation -- The digital philology of interaction heritage](https://www.tandfonline.com/doi/abs/10.1080/09298215.2018.1479428)
> A philologist's approach to heritage is traditionally based on the curation of documents, such as text, audio and video. However, with the advent of interactive multimedia, heritage becomes floating and volatile, and not easily captured in documents. We propose an approach to heritage that goes beyond documents. We consider the crucial role of institutes for interactive multimedia (as motor of a living culture of interaction) and propose that the digital philologist's task will be to promote the collective/shared responsibility of (interactive) documenting, engage engineering in developing interactive approaches to heritage, and keep interaction-heritage alive through the education of citizens.
---
## [MIR Meetup Berlin - Acoustic Fingerprinting in Research](https://0110.be/posts/MIR_Meetup_Berlin_-_Acoustic_Fingerprinting_in_Research.md)
- Published: 2018-04-26T00:00:00Z
- Updated: 2025-11-29T14:40:22Z
- Author: Joren
- ID: 461
- Canonical: https://0110.be/posts/MIR_Meetup_Berlin_-_Acoustic_Fingerprinting_in_Research
- Tags: [Computational musicology](https://0110.be/tags/Computational%20musicology.md), [Presentation](https://0110.be/tags/Presentation.md), [UGent](https://0110.be/tags/UGent.md)
I was kindly invited by SoundCloud to give a presentation on "Acoustic fingerprinting in research". The presentation took place during one of the "MIR Meetups" in Berlin on Monday, April 23, 2018. Before my presentation there was a presentation by Derek and Josh (both SoundCloud engineers) detailing the state of the internal fingerprinting system of SoundCloud.
During my presentation I gave an overview of various applications of acoustic fingerprinting in a music research environment and detailed how these applications can be handled and are implemented in [Panako: an open source fingerprinting system](http://panako.be)
Below the slides used during the presentation can be found:
---
## [Engineering systematic musicology](https://0110.be/posts/Engineering_systematic_musicology.md)
- Published: 2018-02-02T00:00:00Z
- Updated: 2025-11-29T14:41:35Z
- Author: Joren
- ID: 460
- Canonical: https://0110.be/posts/Engineering_systematic_musicology
- Tags: [Presentation](https://0110.be/tags/Presentation.md), [UGent](https://0110.be/tags/UGent.md)
The 11th of January I successfully completed my PhD training under mentorship of Marc Leman with a public defense at [de Krook](http://dekrook.be) in Ghent.
I also handed in my dissertation titled [Engineering systematic musicology: methods and services for computational and empirical music research](https://0110.be/files/attachments/460/2018.01.phd.joren.six-Engineering.systematic.musicology.pdf) ([version of record](https://biblio.ugent.be/publication/8544101)). The dissertation bundles several of [my publications](https://0110.be/Publications) and places them in a framework in the introduction and reflects upon these in the conclusion. The publications all contribute either directly to the field of systematic musicology (e.g. tone scale research) or contributes indirectly by facilitating specific research tasks (e.g. synchronization of multi-modal research data).
The presentation during my defense was meant for a broader audience. During the presentation I gave examples of the research topics I have been working and focused on how these are connected. The [presentation titled Engineering systematic musicology](https://0110.be/phd/presentation/) can be seen by following the previous link and is included below. The slide with the live spectrogram and the slide with the map need to be started by *double clicking* otherwise they remain empty.
I have contributed to the 4th International Digital Libraries for Musicology workshop (DLfM 2017) which was organized in Shanghai, China. It was a satellite event of the ISMIR 2017 conference. Unfortunately I did not mange to find funding to attend the workshop, I did however contribute as co-author to two proceeding papers. Both were presented by Reinier de Valk (thanks again).
### [MIRchiving: Challenges and opportunities of connecting MIR research and digital music archives](https://dl.acm.org/citation.cfm?id=3144755)
By Reinier de Valk (DANS), Anja Volk (Utrecht University), Andre Holzapfel (KTH Royal Institute of Technology) , Aggelos Pikrakis (University of Piraeus), Nadine Kroher (University of Seville - IMUS) and Joren Six (Ghent University - IPEM). Next to the [version of record](https://dl.acm.org/citation.cfm?id=3144755) there is also an author version available of the contribution titled "MIRchiving: Challenges and opportunities of connecting MIR research and digital music archives":\[2017.DLfM.MIRchiving-author.pdf\].
> This study is a call for action for the music information retrieval (MIR) community to pay more attention to collaboration with digital music archives. The study, which resulted from an interdisciplinary workshop and subsequent discussion, matches the demand for MIR technologies from various archives with what is already supplied by the MIR community. We conclude that the expressed demands can only be served sustainably through closer collaborations. Whereas MIR systems are described in scientific publications, usable implementations are often absent. If there is a runnable system, user documentation is often sparse----posing a huge hurdle for archivists to employ it. This study sheds light on the current limitations and opportunities of MIR research in the context of music archives by means of examples, and highlights available tools. As a basic guideline for collaboration, we propose to interpret MIR research as part of a value chain. We identify the following benefits of collaboration between MIR researchers and music archives: new perspectives for content access in archives, more diverse evaluation data and methods, and a more application-oriented MIR research workflow.
### [Applications of duplicate detection: linking meta-data and merging music archives: The experience of the IPEM historical archive of electronic music](https://dl.acm.org/citation.cfm?id=3144759)
By Federica Bressan, Joren Six and Marc Leman (Ghent University - IPEM). Next to the [version of record](https://dl.acm.org/citation.cfm?id=3144759) there is also an author version available of the contribution titled "Applications of duplicate detection: linking meta-data and merging music archives: The experience of the IPEM historical archive of electronic music":\[2017.dlfm_duplicates-author.pdf\].
> This work focuses on applications of duplicate detection for managing digital music archives. It aims to make this mature music information retrieval (MIR) technology better known to archivists and provide clear suggestions on how this technology can be used in practice. More specifically applications are discussed to complement meta-data, to link or merge digital music archives, to improve listening experiences and to re-use segmentation data. The IPEM archive, a digitized music archive containing early electronic music, provides a case study.
The [full DLfM 2017 proceedings](https://dl.acm.org/citation.cfm?id=3144749) are published by ACM.
- [2017.DLfM.MIRchiving-author.pdf](https://0110.be/files/attachments/456/2017.DLfM.MIRchiving-author.pdf)
- [2017.dlfm\_duplicates-author.pdf](https://0110.be/files/attachments/456/2017.dlfm_duplicates-author.pdf)
- [dlfm.png](https://0110.be/files/attachments/456/dlfm.png)
---
## [ESCOM 2017 - Regularity and asynchrony when tapping to tactile, auditory and combined pulses](https://0110.be/posts/ESCOM_2017_-_Regularity_and_asynchrony_when_tapping_to_tactile%2C_auditory_and_combined_pulses.md)
- Published: 2017-07-31T00:00:00Z
- Updated: 2017-08-02T15:40:23Z
- Author: Joren
- ID: 455
- Canonical: https://0110.be/posts/ESCOM_2017_-_Regularity_and_asynchrony_when_tapping_to_tactile%2C_auditory_and_combined_pulses
- Tags: [Presentation](https://0110.be/tags/Presentation.md), [UGent](https://0110.be/tags/UGent.md)
The 25th anniversary edition of the [ESCOM 2017 Conference](http://www.escom2017.org/) conference was organised in August 2017 by the IPEM research group from Ghent University. ESCOM is the conference of the [European Society for the Cognitive Sciences of Music](http://www.escom.org/.I) had two contributions to the conference.
The first was a collaboration with Frank Desmet, Micheline Lesaffre, Nathalie Ehrlé and Séverine Samson. The contribution is titled "Multimodal Analysis of Synchronization Data from Patients with Dementia":\[Desmet-et-al.pdf\]. It details a famework to analyze data in an experiment for patients with dementia.
For the second contribution I was the main researcher. It is the result of a project with students of the systematic musicology course at Ghent University (Laura Arens, Hade Demoor, Thomas Kint) . The contribution is called "Regularity and asynchrony when tapping to tactile, auditory and combined pulses":\[2017.escom_multimodal_async.pdf\]
The presentation "details a multi sensory tapping task":\[six-escom2017-presentation.pdf\] with the aim to develop an assistive technology for dancers.
---
## [AES 2017 - A framework to provide fine-grained time-dependent context for active listening experiences](https://0110.be/posts/AES_2017_-_A_framework_to_provide_fine-grained_time-dependent_context_for_active_listening_experiences.md)
- Published: 2017-06-16T00:00:00Z
- Updated: 2017-07-10T11:16:36Z
- Author: Joren
- ID: 454
- Canonical: https://0110.be/posts/AES_2017_-_A_framework_to_provide_fine-grained_time-dependent_context_for_active_listening_experiences
- Tags: [Code](https://0110.be/tags/Code.md), [Music Information Retrieval](https://0110.be/tags/Music%20Information%20Retrieval.md), [UGent](https://0110.be/tags/UGent.md)
The [2017 AES international conference on semantic audio](http://www.aes.org/conferences/2017/semantic/) was organized at ISS Fraunhofer, Erlangen, Germany. As the birthplace of the MP3 codec, it is holy ground, a stop that can not be skipped on the itinerary of an audio engineers pilgrimage of life. At the conference I presented " A framework to provide fine-grained time-dependent context for active listening experiences":\[2017.author.aes.pdf\] with a poster ("pdf":\[aes_2017_poster.pdf\], "inkscape svg":\[aes_2017_poster_2.svg\]).
On Saturday the eight of April I gave a workshop on the ESP 32 micro controller at [Newline](http://hackerspace.gent/newline), the yearly hackerspace conference of Hackerspace Ghent. The aim was to provide a hands-on introduction. The participants had to program to make the ESP execute the following:
- Blink
- Connecting to wifi
- Sending data from the ESP32
- Sending data using TCP
- Broadcast data over UDP
- Broadcast data using OSC over UDP
- Broadcasting sensor data over UDP and OSC
- Mesh Networking
At the start of the workshop I gave a "presention":\[2017.04.ESP32_intro.pdf\] as an introduction.
---
## [Computational Ethnomusicology: Methodologies for a New Field](https://0110.be/posts/Computational_Ethnomusicology%3A_Methodologies_for_a_New_Field.md)
- Published: 2017-03-30T00:00:00Z
- Updated: 2017-04-03T08:36:34Z
- Author: Joren
- ID: 451
- Canonical: https://0110.be/posts/Computational_Ethnomusicology%3A_Methodologies_for_a_New_Field
- Tags: [Computational ethnomusicology](https://0110.be/tags/Computational%20ethnomusicology.md), [UGent](https://0110.be/tags/UGent.md)
From 27 to 31 March 2017 I have attended a workshop on [Computational Ethnomusicology: Methodologies for a New Field](https://www.lorentzcenter.nl/lc/web/2017/866//info.php3?wsid=866&venue=Oort) at the Lorentz Center in Leiden. The workshop format was aimed at creating time for interdisciplinary discussion.
It was attended by a mix of (Ethno)musicologists, archivists, computer scientists and people identifying themselves as more than one of these categories by varying degrees. This mix ensured a healthy discussion and talks by Frans Wiering, Willard McCarthy, Emilia Gomez, and "may more":\[program.pdf\] provided ample source material to discuss. These discussions ranged from the abstracts around schemata down to concrete of software tools for archive management.
On a more personal side the workshop did provide useful insights to contextualize my research and help form ideas that can be condensed in my PhD dissertation.
---
## [ESP32 Thing as xOSC alternative](https://0110.be/posts/ESP32_Thing_as_xOSC_alternative.md)
- Published: 2017-01-11T00:00:00Z
- Updated: 2025-11-29T15:44:03Z
- Author: Joren
- ID: 449
- Canonical: https://0110.be/posts/ESP32_Thing_as_xOSC_alternative
- Tags: [IPEM](https://0110.be/tags/IPEM.md), [UGent](https://0110.be/tags/UGent.md)
The xOSC board by [x-io technologies](http://x-io.co.uk/x-osc/) looks like a very nice solution in many interactive wireless setups. Judging from the specifications and documentation it offers a lot of value. It is basically a small WiFi transmitter with some sensors and a battery attached to it. The board also has some drawbacks. 1) It is expensive at about € 180. This is especially problematic if you need about five or so for your application.2) It seems that it is also hard to add extra sensors via SPI or I²C. 3)The battery needs to be removed to charge, which makes it harder to build into a fixed enclosure. This post describes an alternative based on the ESP32 platform that addresses these shortcomings.
The ESP32 is a micro-controller with a WiFi transmitter which can be programmed using the Arduino environment. Sparkfun has a thing called the [ESP32 Thing](https://www.sparkfun.com/products/13907) which contains the ESP32 chip. It can be used to build an xOSC alternative.
1. It costs about 20\$, when you add a battery 5\$ and a sensor 20\$ (IMU) you end up with a 45\$ price tag. The price of course depends on which exact sensor/battery you need for your application. A 500mAh lasts about two hours when sending 66 messages per second over WiFi (using UDP).
2. The ESP32 Thing supports the Arduino environment which potentially allows you to use all available Arduino libraries and supported sensors. However, some libraries do contain hardware specific instructions which are often not ported yet. Since the hardware is rather new - large scale production started only 3 months ago - not many libraries have been ported. Fortunately a lot of libraries simply work without any changes. At hackaday they have been testing a few: [ESP32 and Arduino libraries](https://hackaday.com/2016/10/31/whats-new-esp-32-testing-the-arduino-esp32-library/). I had success with the [BNO55](https://learn.adafruit.com/adafruit-bno055-absolute-orientation-sensor/wiring-and-test) library, it did not need any changes. The [OSC library](https://github.com/CNMAT/OSC) did need some small changes to operate as expected.
3. The Thing contains a battery charging circuit. Once embedded into an enclosure the battery can stay in place. The software running on the device even keeps running when changing power sources.
Attached to this post you can find modifications to the Andriod OSC library that enable it to run on the ESP32: [ESP32-Arduino-OSC-library](https://0110.be/files/attachments/449/ESP32-Arduino-OSC-library.zip) together with a patch that sends [random data over OSC](https://0110.be/files/attachments/449/RandomOSC.ino). This should enable you to build an xOSC alternative.
Some drawbacks of the ESP32 is that the supporting software is quite immature. There is a Bluetooth chip on the ESP32 which is currently not supported in the Arduino environment. The setup can be somewhat challenging. The documentation can be improved. Some of the ESP32 Things seem to be unable to connect to old WiFi routers which can be problematic.
- [ESP32-Arduino-OSC-library.zip](https://0110.be/files/attachments/449/ESP32-Arduino-OSC-library.zip)
- [RandomOSC.ino](https://0110.be/files/attachments/449/RandomOSC.ino)
- [ESP32-Thing.jpg](https://0110.be/files/attachments/449/ESP32-Thing.jpg)
---
## [Ipem at Opening Event Digital Week](https://0110.be/posts/Ipem_at_Opening_Event_Digital_Week.md)
- Published: 2016-10-10T00:00:00Z
- Updated: 2025-11-29T19:31:55Z
- Author: Joren
- ID: 448
- Canonical: https://0110.be/posts/Ipem_at_Opening_Event_Digital_Week
- Tags: [UGent](https://0110.be/tags/UGent.md)
Last Saturday, October eight 2016, IPEM was present at the opening event of the [digital week](http://www.digitaleweek.be/). A small video report was made for VRT news, unfortunately our contribution did not make the cut.
I have given a presentation at the the [Newline](https://0x20.be/newline/2016/) conference, a yearly event organized by the [Hackerspace Ghent](http://0x20.be). It was about:
> "In this talk I will give a practical overview on how to connect hard- and software components for musical applications. Next to an overview there will be demos! Do you want to make a musical instrument using a light sensor? Use your smartphone as an input device for a synth? Or are you simply interested in simple low-latency communication between devices? Come to this talk! More concretely the talk will feature the Axoloti audio board, Teensy micro-controller with audio board, MIDI and OSC protocols, Android MIDI features and some sensors."
During the presentation the hard and software components were demonstrated. More concretely an introduction was given to the following:
- [Axoloti audio board](http://www.axoloti.com/): An audio DSP board.
- [Teensy 3.2](https://www.pjrc.com/teensy): A general purpose microcontroller with audio capabilities.
- [TarsosDSP](https://github.com/JorenSix/TarsosDSP): A library for audio processing in Java available for Android.
The "presentation about DIY musical modules":\[2016.03.19.music-interfaces.pdf\] can be downloaded here.
- [2016.03.19.music-interfaces.pdf](https://0110.be/files/attachments/445/2016.03.19.music-interfaces.pdf)
- [logo.png](https://0110.be/files/attachments/445/logo.png)
---
## [Lecture on MIR - Tone Scale Extraction - Acoustic Fingerprinting](https://0110.be/posts/Lecture_on_MIR_-_Tone_Scale_Extraction_-_Acoustic_Fingerprinting.md)
- Published: 2015-10-30T00:00:00Z
- Updated: 2025-11-29T19:35:10Z
- Author: Joren
- ID: 444
- Canonical: https://0110.be/posts/Lecture_on_MIR_-_Tone_Scale_Extraction_-_Acoustic_Fingerprinting
- Tags: [Presentation](https://0110.be/tags/Presentation.md), [UGent](https://0110.be/tags/UGent.md)
This morning, the 30th of October 2015, I gave a lecture on Music Information Retrieval in general and two MIR-tasks in particular. The two more detailed tasks were tone scale analysis and acoustic fingerprinting.
The [Mi Band](http://www.mi.com/in/miband) is a bracelet with some sensors, three RGB leds and a vibration motor. It is marketed as an activity tracker and notifier. It is a neat little device that communicates via Bluetooth LE and has a battery life of around 30 days. It would be nice if it could be used for whatever purpose you want but alas, its API is not very open. This blog post gives pointers to useful resources and tips to make it work with your own code.
There have been some efforts to reverse engineer the Bluetooth protocol. This [blog post](http://allmydroids.blogspot.be/2014/12/xiaomi-mi-band-ble-protocol-reverse.html) contains some info. There are even complete implementations available of the protocol, there is a [Mi Band protocol implementation in python](http://oscaracena.bitbucket.org/mibanda/api/_modules/mibanda.html#BandDevice.setUserInfo) and a [Mi Band protocol implementation in Java](https://github.com/pangliang/miband-sdk-android). It is however not always clear which firmware version is targeted.
I would advise against installing the official Mi Band app, if you want to use it with custom code. The app upgrades the firmware to the latest version and it seems that Xiaomi is obfuscating the protocol more and more with each version. I was able to send vibrate and led commands to a Mi Band with firmware version 10.0.9.3. With the previously mentioned sources and the flow described to the right the device reacts to commands. I used an Android device. The flow:
1. Pair with the Mi Band in the Android Bluetooth setting.
2. In your code, connect to the paired device. Save the device address, you will need it later.
3. Send a pair command to the device. This is part of the Mi Band protocol and has nothing to do with the previous Bluetooth pairing. If all goes well it reacts with a 2. See [here](http://allmydroids.blogspot.be/2014/12/xiaomi-mi-band-ble-protocol-reverse.html)
4. Send user info. This step is crucial and not trivial. The user info needs to be encoded in a certain way and is [CRC'd](https://en.wikipedia.org/wiki/Cyclic_redundancy_check) with the device address. The following is an example implementation of the [Mi Band user info encoding](https://github.com/pangliang/miband-sdk-android/blob/master/src/com/zhaoxiaodan/miband/model/UserInfo.java#L58)
5. Now you can send vibrate or other commands.
Some notes: the self-test command works without the set user step. For Android the [Mi Band protocol implementation in Java](https://github.com/pangliang/miband-sdk-android) works well. To check the firmware version of the device, call the `get device info` characteristic. The last bytes, interpreted as an integer, define the version info. For my device it is 10.9.3.2:
````
Write to characteristic 0000ff05-0000-1000-8000-00805f9b34fb
onCharacteristicWrite status: 0 characteristic 0000ff05-0000-1000-8000-00805f9b34fb
Read firmware version
11 value: 2
12 value: 3
13 value: 9
14 value: 0
15 value: 1
`````
Another note: the set user info needs to be called with a 1 as type the first time the band is used. This is done with `new UserInfo(20111111, 1, 32, 180, 55, "NM", 1)` with the Android sdk by GitHub user pangliang. This sets and overwrites the user info. The next times you do not want to overwrite the info and the type needs to be zero.
- [vibrate\_flowchart\_\_1\_.svg](https://0110.be/files/attachments/439/vibrate_flowchart__1_.svg)
---
## [Synchronizing Multimodal Recordings Using Audio-To-Audio Alignment - In Journal on Multimodal User Interfaces](https://0110.be/posts/Synchronizing_Multimodal_Recordings_Using_Audio-To-Audio_Alignment_-_In_Journal_on_Multimodal_User_Interfaces.md)
- Published: 2015-08-06T00:00:00Z
- Updated: 2025-11-29T19:45:17Z
- Author: Joren
- ID: 434
- Canonical: https://0110.be/posts/Synchronizing_Multimodal_Recordings_Using_Audio-To-Audio_Alignment_-_In_Journal_on_Multimodal_User_Interfaces
- Tags: [Code](https://0110.be/tags/Code.md), [Java](https://0110.be/tags/Java.md), [Music Information Retrieval](https://0110.be/tags/Music%20Information%20Retrieval.md), [Panako](https://0110.be/tags/Panako.md), [Research papers](https://0110.be/tags/Research%20papers.md), [UGent](https://0110.be/tags/UGent.md)
The article titled *"Synchronizing Multimodal Recordings Using Audio-To-Audio Alignment"* by Joren Six and Marc Leman has been accepted for publication in the [Journal on Multimodal User Interfaces](https://www.springer.com/computer/hci/journal/12193). The article will be published later this year. It describes and tests a method to synchronize data-streams. Below you can find the abstract, pointers to the software under discussion and an author version of the article itself.
### Synchronizing Multimodal Recordings Using Audio-To-Audio Alignment
@article{six2015multimodal,
author = {Joren Six and Marc Leman},
title = {{Synchronizing Multimodal Recordings Using Audio-To-Audio Alignment}},
issn = {1783-7677},
volume = {9},
number = {3},
pages = {223-229},
doi = {10.1007/s12193-015-0196-1},
journal = {{Journal of Multimodal User Interfaces}},
publisher = {Springer Berlin Heidelberg},
year = 2015
}
- [2015.synchronized-recording.pdf](https://0110.be/files/attachments/434/2015.synchronized-recording.pdf)
- [SyncDataset.zip](https://0110.be/files/attachments/434/SyncDataset.zip)
- [SyncSink-1.4.jar](https://0110.be/files/attachments/434/SyncSink-1.4.jar)
---
## [Control Audio Time Stretching and Pitch Shifting from Java using Rubber Band And JNI](https://0110.be/posts/Control_Audio_Time_Stretching_and_Pitch_Shifting_from_Java_using_Rubber_Band_And_JNI.md)
- Published: 2015-08-05T00:00:00Z
- Updated: 2025-11-29T19:46:33Z
- Author: Joren
- ID: 437
- Canonical: https://0110.be/posts/Control_Audio_Time_Stretching_and_Pitch_Shifting_from_Java_using_Rubber_Band_And_JNI
- Tags: [Code](https://0110.be/tags/Code.md), [TarsosDSP](https://0110.be/tags/TarsosDSP.md), [UGent](https://0110.be/tags/UGent.md)
This post explains how to do real-time pitch-shifting and audio time-stretching in Java. It uses two components. The first component is [a high quality software C library for audio time-stretching and pitch-shifting C called Rubber Band](http://breakfastquay.com/rubberband/). The second component is a Java audio library called [TarsosDSP](https://github.com/JorenSix/TarsosDSP). To bridge the gap between the two JNI (Java Native Interface) is used. Rubber Band provides a JNI interface and starting from the currently unreleased version 1.8.2, makefiles are provided that make compiling and subsequently using the JNI version of Rubber Band relatively straightforward.
However, it still requires some effort to control real-time pitch-shifting and audio time-stretching from java. To make it more easy some example code and documentation is available in a GitHub repository called [RubberBandJNI](https://github.com/JorenSix/RubberBandJNI). It documents some of the configuration steps needed to get things working. It also offers precompiled libraries and documents how to compile those for the following systems:
- [Audio Time-Stretching vdia JNI on Debian](https://github.com/JorenSix/RubberBandJNI#compiling-the-rubber-band-jni-library-on-debian-linux)
- [Audio Time-Stretching via JNI on Mac OS X](https://github.com/JorenSix/RubberBandJNI#compiling-the-rubber-band-jni-library-on-mac-os-x)
- [Audio Time-Stretching via JNI for Android on Debian](https://github.com/JorenSix/RubberBandJNI#compiling-the-rubber-band-jni-library-on-debian-for-android)
If the instructions are followed rather precisely you are able to control the tempo of a song in real-time with the following Java code:
````java
float tempoFactor = 0.8f;
float pitchFactor = 1.0f;
AudioDispatcher adp = AudioDispatcherFactory.fromPipe("music.mp3", 44100, 4096, 0);
TarsosDSPAudioFormat format = adp.getFormat();
rbs = new RubberBandAudioProcessor(44100, tempoFactor, pitchFactor);
adp.addAudioProcessor(rbs);
adp.addAudioProcessor(new AudioPlayer(JVMAudioInputStream.toAudioFormat(format)));
new Thread(adp).start();
````
---
## [Decode MP3s and other Audio formats the easy way on Android](https://0110.be/posts/Decode_MP3s_and_other_Audio_formats_the_easy_way_on_Android.md)
- Published: 2015-07-14T00:00:00Z
- Updated: 2025-11-29T19:48:15Z
- Author: Joren
- ID: 438
- Canonical: https://0110.be/posts/Decode_MP3s_and_other_Audio_formats_the_easy_way_on_Android
- Tags: [TarsosDSP](https://0110.be/tags/TarsosDSP.md), [UGent](https://0110.be/tags/UGent.md)
This post describes how to decode MP3's using an already compiled ffmpeg binary on android. Using ffmpeg to decode audio on Android has advantages:
- It supports about every audio format known to man. Three channel flac, vorbis with 32 bit samples, ... do not pose a problem.
- Extracting audio from video container formats is supported. Accessing the first audio stream from `mkv`, `avi`, `mov`,... just works.
- Decoding audio frames is more efficient using native code than often buggy Java decoders.
- Resampling and downmixing is supported. If you want to resample incoming audio to e.g. 44.1kHz and only want single channel audio this is easily achievable.
The main disadvantage is that you need an ffmpeg build for your Android device. Luckily some [poor soul](https://github.com/hiteshsondhi88/ffmpeg-android) already managed to compile ffmeg for Android for several architectures. The [precompiled ffmpeg binaries for Android](https://github.com/hiteshsondhi88/ffmpeg-android/releases/download/v0.3.3/prebuilt-binaries.zip) are available for download and are mirrored [here](https://0110.be/releases/TarsosDSP/TarsosDSP-static-ffmpeg/Android/) as well.
To bridge the ffmpeg binary and the java world TarsosDSP contains some glue code. The `AndroidFFMPEGLocator` is responsible to find and extract the correct binary for your Android device. It expects [these ffmpeg binaries](https://0110.be/releases/TarsosDSP/TarsosDSP-static-ffmpeg/Android/) in the assets folder of your Android application. When the correct ffmpeg binary has been extracted and made executable the `PipeDecoder` is able to call it. The `PipeDecoder` calls ffmpeg so that decoded, downmixed and resampled PCM samples are streamed into the Java application via a pipe, which explains its name.
With the TarsosDSP Android library the following code plays an MP3 from external storage:
````java
new AndroidFFMPEGLocator(this);
new Thread(new Runnable() {
@Override
public void run() {
File externalStorage = Environment.getExternalStorageDirectory();
File mp3 = new File(externalStorage.getAbsolutePath(), "/audio.mp3");
AudioDispatcher adp;
adp = AudioDispatcherFactory.fromPipe(mp3.getAbsolutePath(), 44100, 5000, 2500);
adp.addAudioProcessor(new AndroidAudioPlayer(adp.getFormat(), 5000, AudioManager.STREAM_MUSIC));
adp.run();
}
}).start();
````
This code *just works* if the application has the `READ_EXTERNAL_STORAGE` permission, includes a recent TarsosDSP-Android.jar, is ran on one of the supported ffmpeg architectures and has these binaries available in the assets folder.
---
[Newer posts](https://0110.be/Blog.md?page=2)
[Older posts](https://0110.be/Blog.md?page=4)