Visualizing Music Structure

Research Paper Abstract

Understanding a song's structure is crucial to a musician's ability to learn and memorize a song. Discovering, explaining, and sharing structural patterns through visual representation is still done mostly by hand.

This thesis aims to help musicians by providing a tool capable of automatically generating a visualization containing a clear and comprehensive overview of repetition and change of musical aspects such as harmony, timbre, and dynamics. Visualizing these aspects together enables users to navigate the music in a more informed way, and, in the case of music ensembles, helps provide a common view of the music that can be used as a reference.

As opposed to existing methods, this method uses one of the largest growing libraries of music and public music-content data, Spotify, anticipating the decrease in the number of people owning audio files. While previous methods use raw audio as input, Spotify only provides abstracted audio features, making some of the structure extraction more challenging.

Six separate visualizations have been introduced, each showing the structure of a specific musical aspect:

• A repetitive structure visualization, displaying a decomposition of the song into sections, grouped by their harmonic sequential similarity.
• A timbre visualization, showing both segmentation in terms of instrumentation, and gradual timbral changes.
• An event visualization, showing and characterizing moments of timbral anomalies across the song.
• A chord visualization, showing the small scale structure of chord progressions.
• A tonality visualization, showing the large-scale changes in musical key.
• A tempo graph, showing the change, or stability of a song's tempo over time.

Extra care has been taken to ensure the visualizations are accessible, simple, intuitive, and engaging. The approach generalizes across all types of music in order to invite as many people as possible to use them.