Abstract: We present ``abcMLM,'' a masked language model for generating folk-like music that permits a wider range of possibilities compared to autoregressive approaches. It has similar performance to autoregressive approaches overall, albeit with a few peculiarities arising from a different training objective. We demonstrate abcMLM and its different interaction modalities, and discuss its trade-offs in relation to autoregressive modeling.