P1-04: A Review of Validity and Its Relationship to Music Information Research
Bob L. T. Sturm (KTH Royal Institute of Technology), Arthur Flexer (Johannes Kepler University Linz)*
Subjects (starting with primary): Evaluation, datasets, and reproducibility ; Philosophical and ethical discussions ; Evaluation, datasets, and reproducibility -> evaluation methodology ; Philosophical and ethical discussions -> philosophical and methodological foundations
Presented In Person: 4-minute short-format presentation
Validity is the truth of an inference made from evidence and is a central concern in scientific work. Given the maturity of the domain of music information research (MIR), validity in our opinion should be discussed and considered much more than it has been so far. Puzzling MIR phenomena like adversarial attacks, horses, and performance glass ceilings become less mysterious through the lens of validity. In this paper, we review the subject of validity as presented in a key reference of causal inference: Shadish et al., Experimental and Quasi-experimental Designs for Generalised Causal Inference [1]. We discuss the four types of validity and threats to each one. We consider them in relationship to MIR experiments grounded with a practical demonstration using a typical MIR experiment.
If the video does not load properly please use the direct link to video