gladia-research-group/phalar

PHALAR is a research framework for teaching a machine learning model to understand music. The goal is to produce "embeddings", compact numerical representations of audio that capture what a piece of music sounds like and how it relates to other pieces. These representations can then be used for tasks like identifying chords, tracking the beat, or comparing individual instrument tracks within a recording. The project's central contribution combines two ideas. First, it applies a learned spectral pooling technique, a way of processing sound by analyzing how different frequencies relate over time. Second, it feeds the result into a phase-equivariant complex-valued neural network, designed to remain sensitive to the phase of audio signals. Phase refers to timing relationships between sound waves, many audio models discard this information, but the authors argue it matters for music understanding. The model is trained through self-supervised contrastive learning, meaning it learns from unlabeled music data by comparing similar and dissimilar audio clips rather than requiring manually labeled examples. Once trained, the embeddings can be applied to downstream tasks like chord detection, beat tracking, and stem similarity evaluation. The repository includes the official model implementation, comparison baselines, pre-trained checkpoints, and training configuration files. The code is written in Python 3.12 with PyTorch and includes optional integration with FluidSynth for synthesized audio generation. It is the official implementation accompanying a paper published at the Forty-Third International Conference on Machine Learning. The project is released under the MIT license.