Automatic SATB Singing Voice Classification Using Statistical Pseudo-Labelling and CNN-LSTM
Keywords:
SATB classification, Pseudo-labelling, Fundamental Frequency, Mel-spectrogram, CNN-LSTMAbstract
Accurate classification of singers into Soprano, Alto, Tenor, and Bass (SATB) voice categories is essential for choir performance and vocal health. However, existing automatic SATB classification methods are constrained by the limited availability of labeled datasets. This study proposes a statistically guided pseudo-labeling framework to expand labeled singing voice data and improve automatic SATB classification. Fundamental frequency (F0) was extracted from professionally labeled recordings in the Choral Singing Dataset (CSD) and ESMUC Choir Dataset using the YIN algorithm. Ten F0-derived statistical features were evaluated through statistical significance testing, effect size analysis, overlap analysis, and single-feature classification to identify the most discriminative feature for threshold-based pseudo-label generation. The selected feature was then used to assign SATB pseudo-labels to 3,093 VocalSet recordings, producing a combined dataset of 3,188 labeled recordings. The expanded dataset was segmented into four-second audio chunks, converted into Mel-spectrograms, and used to train and compare Convolutional Neural Network (CNN) and CNN–Long Short-Term Memory (CNN-LSTM) models. Experimental results identified the 75th percentile (P75) of F0 as the most discriminative statistic, achieving 98.95% single-feature classification accuracy. On the internal test set, the CNN-LSTM model outperformed the baseline CNN, achieving 94.6% accuracy and a macro F1-score of 0.92, compared with 92.6% accuracy and a macro F1-score of 0.90 for CNN. External evaluation on previously unseen YouTube recordings further demonstrated superior generalization, with CNN-LSTM achieving 65.4% accuracy versus 42.5% for CNN. These findings demonstrate that statistically guided pseudo-labeling effectively expands labeled training data and that integrating temporal modeling significantly improves automatic SATB singing voice classification.
Downloads
References
Bashir, R. N., Shahid, M. A., Rashid, T., Faheem, M., Saidani, T., Saidani, O., & Khan, A. R. (2025). Voice pathology identification using mel spectrogram features and deep learning. Signal, Image and Video Processing, 19(11), 909. https://doi.org/10.1007/s11760-025-04527-4
Chandna, P., Cuesta, H., Petermann, D., & Gómez, E. (2022). A Deep-Learning Based Framework for Source Separation, Analysis, and Synthesis of Choral Ensembles. Frontiers in Signal Processing, 2(April), 1–15. https://doi.org/10.3389/frsip.2022.808594
Cochoy, J. (2023). Human Voice Pitch Estimation: A Convolutional Network with Auto-Labeled and Synthetic Data. 1–11. http://arxiv.org/abs/2308.07170
Cuesta, H., & Gómez, E. (2022a). ESMUC Choir Dataset. Zenodo. https://doi.org/10.5281/zenodo.5848990
Cuesta, H., & Gómez, E. (2022b). Voice Assignment in Vocal Quartets Using Deep Learning Models Based on Pitch Salience. Transactions of the International Society for Music Information Retrieval, 5(1), 99. https://doi.org/10.5334/tismir.121
Cuesta, H., Gómez, E., Martorell, A., & Loáiciga, F. (2019). Choral Singing Dataset. Zenodo. https://doi.org/10.5281/zenodo.2649950
de Cheveigné, A., & Kawahara, H. (2002). YIN, a fundamental frequency estimator for speech and music. The Journal of the Acoustical Society of America, 111(4), 1917–1930. https://doi.org/10.1121/1.1458024
Frizzell, E. Y., & Windsor, L. C. (2021). Effects of teaching experience and culture on choral directors’ descriptions of choral tone. PLOS ONE, 16(12), e0256587. https://doi.org/10.1371/journal.pone.0256587
Lycke, H., De Jong, F., Decoster, W., & Van Hulle, M. (2013). Identification of three natural voice Groups By Phonetography - A data driven approach. https://www.researchgate.net/profile/Hugo-Lycke/publication/364029030_Voice_Classification/links/63369eaf769781354ea83bc2/Voice-Classification.pdf
Meerschman, I., D’haeseleer, E., Cammu, H., Kissel, I., Papeleu, T., Leyns, C., Daelman, J., Dannhauer, J., Vanden Abeele, L., Konings, V., Demarbaix, E., & Van Lierde, kristiane. (2025). Voice Quality of Choir Singers and the Effect of a Performance on the Voice. Journal of Voice, 39(2), 561.e1-561.e10. https://doi.org/10.1016/j.jvoice.2022.08.017
Müller, M., Wang, Z., Caffier, F., & Caffier, P. P. (2022). New objective timbre parameters for classification of voice type and fach in professional opera singers. Scientific Reports, 12(1), 17921. https://doi.org/10.1038/s41598-022-22821-w
Pratama, K. B., Suyanto, S., & Rachmawati, E. (2021). Human Vocal Type Classification using MFCC and Convolutional Neural Network. 2021 International Conference on Communication & Information Technology (ICICT), 43–48. https://doi.org/10.1109/ICICT52195.2021.9568474
Van, T. P., Quang, N. T. N., & Thanh, T. M. (2019). Deep Learning Approach for Singer Voice Classification of Vietnamese Popular Music. Proceedings of the Tenth International Symposium on Information and Communication Technology - SoICT 2019, 255–260. https://doi.org/10.1145/3368926.3369700
Vishwakarma, H., Lin, H., Sala, F., & Vinayak, R. K. (2024). Promises and Pitfalls of Threshold-based Auto-labeling. http://arxiv.org/abs/2211.12620
Wilkins, J., Wahl, A., Pardo, B., & Science, C. (2018). VocalSet: a singing voice dataset. 468–474. https://doi.org/10.5281/zenodo.1193957
Zaman, K., Sah, M., Direkoglu, C., & Unoki, M. (2023). A Survey of Audio Classification Using Deep Learning. IEEE Access, 11(September), 106620–106649. https://doi.org/10.1109/ACCESS.2023.3318015
