Research area · 2005–2019
Can we automatically discover how to describe a sound?
Audio analysis by artificial intelligence. Describing the audio signal through formulas — descriptors — requires knowledge of signal processing; the line addresses the automatic discovery of descriptors and the improvement, through search, of already-trained sound classifiers. It begins with Giordano Cabral's PhD, with Sony CSL's EDS, includes HarmIn's harmonic analysis and the sound classifiers for public safety at UFRPE, and a study on the progress of the field itself (2016).
01 · AI-based audio analysis: music information retrieval
Contributions
EDS in chord recognition · 2005–2006
Sony CSL's EDS (Extractor Discovery System) proposes to automatically discover audio descriptors. With François Pachet and Jean-Pierre Briot, Cabral carried out the first part of the validation: Automatic X Traditional Descriptor Extraction: the Case of Chord Recognition (ISMIR 2005, London; zenodo.1415701, hal-01416436) compares classical approaches with the system operated by an inexperienced user, building a guitar chord recognizer. The follow-up is Recognizing Chords with EDS: Part One (CMMR 2005, Pisa; LNCS 3902, Springer, 2006; doi.org/10.1007/11751069_17, hal-01336914). The companion work from the same year is Impact of Distance in Pitch Class Profile Computation (10th SBCM 2005; hal-01416428, PDF in the proceedings). It is the first work of the doctorate presented at an international conference.
Analytical descriptors for harmony and rhythm · 2007
With the IMPA team — Sergio Krakowski and Luiz Velho — and with Pachet, Pierre Roy and Briot: Analytical Features to Extract Harmonic or Rhythmic Information (11th SBCM 2007; hal-01305977, PDF in the proceedings) and the technical report Some Case Studies in Automatic Descriptor Extraction (Rio de Janeiro, 2007).
HarmIn · 2007
A tool for analyzing harmonic progressions, applied to Tom Jobim's work using the Cancioneiro Jobim, with musicologist Robert Willey (University of Louisiana at Lafayette). The progressions become a graph of chord transitions arranged on the circle of fifths, with the thickness of each edge proportional to its frequency in the repertoire — a composer's recurring harmonic paths become visible. Analyzing Harmonic Progressions with HarmIn: the Music of Antonio Carlos Jobim received the best student technical paper award at the 11th SBCM (São Paulo, September 1–3, 2007), according to the symposium website; the Lattes entry records "Best paper prize". PDF in the proceedings. In 2026, the tool returned as a portable skill that builds a transition matrix on the circle of fifths for any requested artist.
Audio as game input · 2008–2010
Mapping Sounds into Commands (AES 35th International Conference: Audio for Games, London, 2008) and The Acoustick: Game Command Extraction from Audio Input Stream (SBGames 2010), with Roberto Cássio Júnior. Both are held in collections without publicly accessible links.
Audio Alerta — audio monitoring for public safety · 2010–2014
At UFRPE, the problem shifted to automatically detecting gunshots, screams, impacts, ambient sound and silence from microphone input. This was the focus of the CNPq Technological Development Productivity Fellowship (2011–2014), the CNPq project "Audio Monitoramento para Segurança Pública" (2010–2013) and the FACEPE project "Classificação Automática de Áudio para Monitoramento de Espaços Públicos" (call 11/2011, 2012–2014). The work produced Dalton Francisco de Araújo's master's thesis, Busca como sistema de apoio à melhoria de classificadores automáticos de áudio (UFRPE, 2012, FACEPE scholarship), and the software Audio Alerta (2012) and Audio Alerta — segurança privada (2013), with Márcio Dahia, Roberto Cássio Júnior, Ivo Frazão and Raphael Holanda.
Evolutionary generation of acoustic features · 2014–2019
The productivity fellowship project "Meta-Busca em Classificadores de Áudio e de Dados para Segurança, Entretenimento e Educação" (2014–) explored generating acoustic features instead of selecting them manually. With José Antonio Alves Menezes and Bruno Gomes, the resulting journal article is Feature Learning with Multi-objective Evolutionary Computation in the generation of Acoustic Features (Inteligencia Artificial — IBERAMIA, vol. 22, no. 64, 2019; doi.org/10.4114/intartif.vol22iss64pp14-35).
Measuring progress in MIREX · 2016
MIREX has been the evaluation arena for music information retrieval since 2005. With Ricardo Scholz and Geber Ramalho, the work compiles six years of results, proposes a configurable index of evolution trend by task, and discusses hypotheses for the stagnation of some of them: Cross Task Study on MIREX Recent Results: an Index for Evolution Measurement and Some Stagnation Hypotheses (ISMIR 2016, New York; zenodo.1416898). It is the only work in the curriculum on the meta-science of the field itself.
Chorus, transcription and podcasts · 2014–2021
Chorus detection based on intensity, with Renato Celso Santos Rodrigues (15th SBCM, 2015; master's thesis co-supervised in 2016); chord transcription post-processing, with Uraquitan Cunha (12th AES Brasil, 2014); predicting podcast popularity from metadata, content and audio, a master's thesis by Luiz Delando Santos Moreira Júnior (2021).
02 · AI-based audio analysis: music information retrieval
Where the work continues
The 2021 Chord Detection Challenge, with Moises.ai, and the course Computação Musical, in which music information retrieval is one of the four syllabus units.
