How can an AI system generate a musician's accompaniment live?

D'Accord Guitar · 2001

The problem was going from chord chart to execution: choosing the positions on the guitar neck, modeling the guitarist's hand and animating the performance for someone learning. The system came out of the project "Intelligent Agents for Complex Environments: study of real-time musical accompaniment" (1999–2001, led by Geber Ramalho) and gave rise to Giordano Cabral's master's thesis at CIn, D'Accord Guitar: um Sistema de Execução Violonística (D'Accord Guitar: A Guitar Performance System) (2002, advisor Geber Ramalho, CNPq scholarship). Four papers describe the system, with Izabel Zanforlin, Hugo Santana, Rodrigo Lima and Geber Ramalho: Da Cifra para o Braço (From Chord Chart to Fretboard) and Playing Along with D'Accord Guitar (VIII SBCM 2001, Fortaleza), Modelagem da Execução Musical em Violão e Guitarra (Modeling Musical Performance on Acoustic and Electric Guitar) (ENIA 2001) and D'Accord Guitar: an Innovative Guitar Performance System (Journées d'Informatique Musicale, Bourges, 2001; hal-02987455). The D'Accord Violão Player software (2001) and the company D'Accord Music Software came out of it. The SBCM PDFs are in the proceedings: research and discussion.

Pattern-based rhythmic accompaniment · 2004

Using Patterns to Generate Rhythmic Accompaniment for Guitar generates accompaniment from patterns extracted from real performances. The team included an ethnomusicologist, Carlos Sandroni, alongside Márcio Dahia, Hugo Santana, Ernesto Trajano and Geber Ramalho. Sound and Music Computing / JIM 2004, Paris; hal-03354323, zenodo.849240. The work precedes the doctoral thesis and builds on D'Accord Guitar.

Incremental parsing · 2005–2006

With Jean-Pierre Briot and François Pachet, Cabral adapted the IP algorithm — real-time continuation of musical phrases — to accompaniment systems: Parsing incremental para acompanhamento em tempo real (Incremental Parsing for Real-Time Accompaniment) (10th SBCM, Belo Horizonte, 2005; hal-01416429, PDF in the proceedings) and Incremental Parsing for Real-Time Accompaniment Systems (FLAIRS 2006, AI in music and art track; hal-01338563, PDF).

Harmonisation automatique en temps réel — the thesis · 2008

PhD at LIP6, Université Pierre et Marie Curie (Paris 6), SMA team — Systèmes Multi-Agents, from 2003 to 2008, with a CAPES scholarship. Co-supervised by Jean-Pierre Briot (LIP6) and François Pachet (Sony Computer Science Laboratory, Paris); from 2003 to 2007 the work was carried out alongside the Sony CSL team. The thesis defines the système d'harmonisation automatique en temps réel (real-time automatic harmonization system): an application that accompanies a melody, possibly improvised, finding the appropriate harmony over a fixed rhythm, with partial information and mutual influence between system and user. The method extends music information retrieval techniques to work directly on the acoustic signal, in experiments per development cycle; the result is a framework of solutions evaluated against human musicians, and a simple solution outperformed all the others, including the more complex ones. Defense on 09/07/2008, très honorable distinction; committee of Jean-Pierre Briot, François Pachet, Myriam Desainte-Catherine and Sergi Jordà (reviewers), Jean-François Perrot and Geber Ramalho. Record NNT 2008PA066236 at theses.fr; abstract and committee at the LIP6 page. No open-access PDF available.

Ramin · 2008–2009

The thesis framework acquired a name and a software implementation: Ramin: A Framework for Real-Time Improvised Accompaniment and Harmonization (IEEE International Symposium on Multimedia, Berkeley, 2008) and the program "Ramin, um Sistema de Harmonização em Tempo Real" (2008), listed on Lattes. The following year, Comparing Techniques for the Automatic Accompaniment of Audio Improvisation (IEEE ISM 2009, San Diego), with Briot, Pachet, and Ramalho, compared the accompaniment techniques. Both papers appear in IEEE proceedings that are not open access, with no open-access link.

Capturing a Musician's Groove · 2015

Seven years after the thesis, collaboration with Sony CSL resumed with a demonstration at IJCAI 2015 in Buenos Aires: Capturing a Musician's Groove: Generation of Realistic Accompaniments from Single Song Recordings, with Mathieu Ramona and François Pachet. A concatenative synthesis engine takes a recording of a performer playing a song and uses that material to generate accompaniment for any other song. Particular attention is paid to timing deviations at the boundaries of the extracted segments, which create the groove: accidental deviations are corrected and intentional ones preserved. Abstract at IJCAI · PDF.

Brazyle · 2017

Software registered with Lattes together with François Pachet: a Brazilian-style database linked to Sony CSL's Flow Machines project. Daccord states, on its institutional page, that the database "is used in the Flow Machines project". Flow Machines produced the album Hello World (2018); on the album's credits page, Cabral's name appears in the acknowledgments list; details are in press.

Chord Detection Challenge · 2021

An open chord detection challenge organized with Moises.ai, launched on March 16, 2021, with results in June (CIn portal, June 9, 2021). The paper evaluating the chord estimation and alignment tasks was published at SBCM 2021, with Valter Jorge da Silva (doi.org/10.5753/sbcm.2021.19425).

Cabral co-advised Uraquitan Sidney Cunha's doctoral research on automatic chord transcription (2010 and 2015 records in Lattes) and advised the undergraduate theses of Arthur Lubambo Peixoto Accioly (chord transcription with hybrid sources, 2017) and Bruno Tavares de Melo Gomes (chord following with dynamic time warping, 2018).

The related product line is iChords: a melody and harmony recognition engine (RHAE/CNPq, 2008–2010), iChords.Net (2008–2010), iChords Station (2011) and the app (2019), with the Banco do Nordeste and FACEPE/Tecnova projects of 2019, under the responsibility of Rodrigo Pereira. It is technological development at Daccord, of which Cabral is a founding partner.

Continues in: Music information retrieval and in the course Computação Musical (Music Computing).