P5-13: Online Symbolic Music Alignment With Offline Reinforcement Learning
Silvan Peter (JKU)*
Subjects (starting with primary): MIR tasks -> alignment, synchronization, and score following ; MIR fundamentals and methodology -> symbolic music processing
Presented In Person: 4-minute short-format presentation
Symbolic Music Alignment is the process of matching
performed MIDI notes to corresponding score notes. In
this paper, we introduce a reinforcement learning (RL)-
based online symbolic music alignment technique. The
RL agent — an attention-based neural network — itera-
tively estimates the current score position from local score
and performance contexts. For this symbolic alignment
task, environment states can be sampled exhaustively and
the reward is dense, rendering a formulation as a simpli-
fied offline RL problem straightforward. We evaluate the
trained agent in three ways. First, in its capacity to identify
correct score positions for sampled test contexts; second,
as the core technique of a complete algorithm for symbolic
online note-wise alignment; and finally, as a real-time sym-
bolic score follower. We further investigate the pitch-based
score and performance representations used as the agent’s
inputs. To this end, we develop a second model, a two-
step Dynamic Time Warping (DTW)-based offline align-
ment algorithm leveraging the same input representation.
The proposed model outperforms a state-of-the-art refer-
ence model of offline symbolic music alignment.
If the video does not load properly please use the direct link to video