benchmark.md

Model Benchmarks

Models Evaluated

hubert-base-ls960, hubert-large-ls960-ft, wav2vec2-base, wav2vec2-large-960h, wav2vec2-large-lv60, wav2vec2-xls-r-1b, wav2vec2-xls-r-300m, wavlm-base-plus, wavlm-large, whisper-large, whisper-small

Afrikaans (af)

AfrikaansEmotionalSpeechCorpus

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.65	0.67	✅
2	hubert-large-ls960-ft	0.32	0.41	✅
3	wav2vec2-base	0.57	0.66	✅
4	wav2vec2-large-960h	0.38	0.43	✅
5	wav2vec2-large-lv60	0.36	0.40	✅
6	wav2vec2-xls-r-1b	0.35	0.24	✅
7	wav2vec2-xls-r-300m	0.38	0.37	✅
8	wavlm-base-plus	0.53	0.59	✅
9	wavlm-large	0.29	0.63	✅
10	whisper-large	0.68	0.65	✅
11	whisper-small	0.75	0.71	✅

Amharic (am)

ASED

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.88	0.89	✅
2	hubert-large-ls960-ft	0.64	0.63	✅
3	wav2vec2-base	0.75	0.75	✅
4	wav2vec2-large-960h	0.30	0.32	✅
5	wav2vec2-large-lv60	0.23	0.27	✅
6	wav2vec2-xls-r-1b	0.35	0.40	✅
7	wav2vec2-xls-r-300m	0.31	0.32	✅
8	wavlm-base-plus	0.87	0.84	✅
9	wavlm-large	0.80	0.82	✅
10	whisper-large	0.88	0.86	✅
11	whisper-small	0.91	0.90	✅

Arabic (ar)

ANAD

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.63	0.58	✅
2	hubert-large-ls960-ft	0.64	0.66	✅
3	wav2vec2-base	0.62	0.57	✅
4	wav2vec2-large-960h	0.62	0.65	✅
5	wav2vec2-large-lv60	0.64	0.64	✅
6	wav2vec2-xls-r-1b	0.61	0.63	✅
7	wav2vec2-xls-r-300m	0.61	0.61	✅
8	wavlm-base-plus	0.66	0.61	✅
9	wavlm-large	0.66	0.71	✅
10	whisper-large	0.78	0.82	✅
11	whisper-small	0.72	0.79	✅

Bengali (bn)

BANSpEmo

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.52	0.41	✅
2	hubert-large-ls960-ft	0.24	0.23	✅
3	wav2vec2-base	0.41	0.32	✅
4	wav2vec2-large-960h	0.23	0.23	✅
5	wav2vec2-large-lv60	0.26	0.25	✅
6	wav2vec2-xls-r-1b	0.19	0.26	✅
7	wav2vec2-xls-r-300m	0.25	0.23	✅
8	wavlm-base-plus	0.49	0.35	✅
9	wavlm-large	0.41	0.40	✅
10	whisper-large	0.35	0.40	✅
11	whisper-small	0.30	0.38	✅

SUBESCO

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.77	0.78	✅
2	hubert-large-ls960-ft	0.56	0.54	✅
3	wav2vec2-base	0.69	0.68	✅
4	wav2vec2-large-960h	0.24	0.24	✅
5	wav2vec2-large-lv60	0.28	0.27	✅
6	wav2vec2-xls-r-1b	0.36	0.38	✅
7	wav2vec2-xls-r-300m	0.51	0.48	✅
8	wavlm-base-plus	0.76	0.78	✅
9	wavlm-large	0.76	0.74	✅
10	whisper-large	0.88	0.89	✅
11	whisper-small	0.88	0.89	✅

German (de)

EmoDB

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.75	0.78	✅
2	hubert-large-ls960-ft	0.43	0.44	✅
3	wav2vec2-base	0.68	0.70	✅
4	wav2vec2-large-960h	0.31	0.34	✅
5	wav2vec2-large-lv60	0.30	0.26	✅
6	wav2vec2-xls-r-1b	0.33	0.30	✅
7	wav2vec2-xls-r-300m	0.33	0.35	✅
8	wavlm-base-plus	0.71	0.70	✅
9	wavlm-large	0.52	0.53	✅
10	whisper-large	0.84	0.86	✅
11	whisper-small	0.82	0.84	✅

Greek (el)

AESDD

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.69	0.75	✅
2	hubert-large-ls960-ft	0.43	0.32	✅
3	wav2vec2-base	0.58	0.54	✅
4	wav2vec2-large-960h	0.37	0.32	✅
5	wav2vec2-large-lv60	0.28	0.25	✅
6	wav2vec2-xls-r-1b	0.21	0.26	✅
7	wav2vec2-xls-r-300m	0.38	0.36	✅
8	wavlm-base-plus	0.62	0.63	✅
9	wavlm-large	0.70	0.57	✅
10	whisper-large	0.57	0.56	✅
11	whisper-small	0.60	0.57	✅

English (en)

IESC

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.82	0.82	✅
2	hubert-large-ls960-ft	0.45	0.53	✅
3	wav2vec2-base	0.70	0.66	✅
4	wav2vec2-large-960h	0.28	0.31	✅
5	wav2vec2-large-lv60	0.21	0.25	✅
6	wav2vec2-xls-r-1b	0.28	0.24	✅
7	wav2vec2-xls-r-300m	0.26	0.25	✅
8	wavlm-base-plus	0.70	0.78	✅
9	wavlm-large	0.76	0.81	✅
10	whisper-large	0.79	0.74	✅
11	whisper-small	0.78	0.71	✅

RAVDESS

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.65	0.65	✅
2	hubert-large-ls960-ft	0.34	0.35	✅
3	wav2vec2-base	0.51	0.59	✅
4	wav2vec2-large-960h	0.13	0.15	✅
5	wav2vec2-large-lv60	0.23	0.27	✅
6	wav2vec2-xls-r-1b	0.22	0.23	✅
7	wav2vec2-xls-r-300m	0.31	0.34	✅
8	wavlm-base-plus	0.68	0.69	✅
9	wavlm-large	0.66	0.55	✅
10	whisper-large	0.75	0.68	✅
11	whisper-small	0.70	0.61	✅

Spanish (es)

MESD

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.63	0.58	✅
2	hubert-large-ls960-ft	0.33	0.26	✅
3	wav2vec2-base	0.38	0.27	✅
4	wav2vec2-large-960h	0.32	0.26	✅
5	wav2vec2-large-lv60	0.15	0.13	✅
6	wav2vec2-xls-r-1b	0.27	0.24	✅
7	wav2vec2-xls-r-300m	0.29	0.24	✅
8	wavlm-base-plus	0.40	0.28	✅
9	wavlm-large	0.51	0.41	✅
10	whisper-large	0.69	0.67	✅
11	whisper-small	0.67	0.68	✅

Persian (fa)

SHEMO

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.77	0.80	✅
2	hubert-large-ls960-ft	0.57	0.58	✅
3	wav2vec2-base	0.71	0.76	✅
4	wav2vec2-large-960h	0.44	0.44	✅
5	wav2vec2-large-lv60	0.43	0.43	✅
6	wav2vec2-xls-r-1b	0.60	0.67	✅
7	wav2vec2-xls-r-300m	0.44	0.46	✅
8	wavlm-base-plus	0.77	0.79	✅
9	wavlm-large	0.72	0.80	✅
10	whisper-large	0.80	0.82	✅
11	whisper-small	0.79	0.83	✅

French (fr)

CaFE

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.64	0.60	✅
2	hubert-large-ls960-ft	0.33	0.33	✅
3	wav2vec2-base	0.46	0.38	✅
4	wav2vec2-large-960h	0.26	0.24	✅
5	wav2vec2-large-lv60	0.20	0.21	✅
6	wav2vec2-xls-r-1b	0.26	0.25	✅
7	wav2vec2-xls-r-300m	0.29	0.25	✅
8	wavlm-base-plus	0.53	0.53	✅
9	wavlm-large	0.72	0.70	✅
10	whisper-large	0.49	0.53	✅
11	whisper-small	0.55	0.61	✅

Hindi (hi)

Hindi-Dataset

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.72	0.70	✅
2	hubert-large-ls960-ft	0.33	0.31	✅
3	wav2vec2-base	0.62	0.61	✅
4	wav2vec2-large-960h	0.22	0.23	✅
5	wav2vec2-large-lv60	0.20	0.20	✅
6	wav2vec2-xls-r-1b	0.35	0.35	✅
7	wav2vec2-xls-r-300m	0.22	0.22	✅
8	wavlm-base-plus	0.70	0.65	✅
9	wavlm-large	0.69	0.70	✅
10	whisper-large	0.67	0.69	✅
11	whisper-small	0.68	0.71	✅

Hungarian (hu)

HungarianEmotionalSpeechCorpus

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.54	0.52	✅
2	hubert-large-ls960-ft	0.29	0.21	✅
3	wav2vec2-base	0.42	0.43	✅
4	wav2vec2-large-960h	0.28	0.24	✅
5	wav2vec2-large-lv60	0.22	0.23	✅
6	wav2vec2-xls-r-1b	0.19	0.22	✅
7	wav2vec2-xls-r-300m	0.28	0.23	✅
8	wavlm-base-plus	0.52	0.51	✅
9	wavlm-large	0.45	0.49	✅
10	whisper-large	0.48	0.44	✅
11	whisper-small	0.41	0.52	✅

Indonesian (id)

IndoWaveSentiment

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.97	0.99	✅
2	hubert-large-ls960-ft	0.63	0.46	✅
3	wav2vec2-base	0.78	0.77	✅
4	wav2vec2-large-960h	0.61	0.37	✅
5	wav2vec2-large-lv60	0.50	0.57	✅
6	wav2vec2-xls-r-1b	0.63	0.59	✅
7	wav2vec2-xls-r-300m	0.58	0.46	✅
8	wavlm-base-plus	0.94	0.94	✅
9	wavlm-large	0.97	0.94	✅
10	whisper-large	0.85	0.87	✅
11	whisper-small	0.96	0.97	✅

Italian (it)

EMOVO

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.66	0.57	✅
2	hubert-large-ls960-ft	0.26	0.28	✅
3	wav2vec2-base	0.47	0.56	✅
4	wav2vec2-large-960h	0.26	0.28	✅
5	wav2vec2-large-lv60	0.21	0.26	✅
6	wav2vec2-xls-r-1b	0.31	0.24	✅
7	wav2vec2-xls-r-300m	0.25	0.25	✅
8	wavlm-base-plus	0.69	0.58	✅
9	wavlm-large	0.64	0.55	✅
10	whisper-large	0.78	0.70	✅
11	whisper-small	0.78	0.64	✅

Emozionalmente

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.72	0.71	✅
2	hubert-large-ls960-ft	0.46	0.49	✅
3	wav2vec2-base	0.62	0.61	✅
4	wav2vec2-large-960h	0.30	0.32	✅
5	wav2vec2-large-lv60	0.28	0.30	✅
6	wav2vec2-xls-r-1b	0.26	0.28	✅
7	wav2vec2-xls-r-300m	0.46	0.47	✅
8	wavlm-base-plus	0.75	0.72	✅
9	wavlm-large	0.69	0.73	✅
10	whisper-large	0.79	0.80	✅
11	whisper-small	0.77	0.75	✅

Japanese (ja)

JVNV

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.77	0.86	✅
2	hubert-large-ls960-ft	0.46	0.49	✅
3	wav2vec2-base	0.61	0.67	✅
4	wav2vec2-large-960h	0.33	0.39	✅
5	wav2vec2-large-lv60	0.31	0.26	✅
6	wav2vec2-xls-r-1b	0.40	0.34	✅
7	wav2vec2-xls-r-300m	0.42	0.43	✅
8	wavlm-base-plus	0.75	0.84	✅
9	wavlm-large	0.87	0.85	✅
10	whisper-large	0.83	0.87	✅
11	whisper-small	0.80	0.81	✅

Kazakh (kk)

KazakhEmotionalTTS

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.54	0.51	✅
2	hubert-large-ls960-ft	0.28	0.29	✅
3	wav2vec2-base	0.52	0.53	✅
4	wav2vec2-large-960h	0.22	0.24	✅
5	wav2vec2-large-lv60	0.21	0.21	✅
6	wav2vec2-xls-r-1b	0.30	0.32	✅
7	wav2vec2-xls-r-300m	0.37	0.35	✅
8	wavlm-base-plus	0.52	0.51	✅
9	wavlm-large	0.58	0.58	✅
10	whisper-large	0.70	0.65	✅
11	whisper-small	0.70	0.65	✅

Kannada (kn)

Kannada-Dataset

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.21	0.29	✅
2	hubert-large-ls960-ft	0.19	0.19	✅
3	wav2vec2-base	0.18	0.22	✅
4	wav2vec2-large-960h	0.16	0.27	✅
5	wav2vec2-large-lv60	0.18	0.23	✅
6	wav2vec2-xls-r-1b	0.15	0.26	✅
7	wav2vec2-xls-r-300m	0.26	0.17	✅
8	wavlm-base-plus	0.19	0.26	✅
9	wavlm-large	0.36	0.48	✅
10	whisper-large	0.37	0.36	✅
11	whisper-small	0.41	0.30	✅

Korean (ko)

KESDy18

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.76	0.80	✅
2	hubert-large-ls960-ft	0.46	0.46	✅
3	wav2vec2-base	0.67	0.70	✅
4	wav2vec2-large-960h	0.25	0.23	✅
5	wav2vec2-large-lv60	0.25	0.24	✅
6	wav2vec2-xls-r-1b	0.43	0.43	✅
7	wav2vec2-xls-r-300m	0.40	0.39	✅
8	wavlm-base-plus	0.70	0.74	✅
9	wavlm-large	0.77	0.76	✅
10	whisper-large	0.78	0.79	✅
11	whisper-small	0.81	0.81	✅

Odia (or)

SITB-OSED

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.76	0.75	✅
2	hubert-large-ls960-ft	0.61	0.62	✅
3	wav2vec2-base	0.65	0.64	✅
4	wav2vec2-large-960h	0.44	0.43	✅
5	wav2vec2-large-lv60	0.29	0.28	✅
6	wav2vec2-xls-r-1b	0.54	0.58	✅
7	wav2vec2-xls-r-300m	0.50	0.51	✅
8	wavlm-base-plus	0.79	0.78	✅
9	wavlm-large	0.82	0.83	✅
10	whisper-large	0.84	0.83	✅
11	whisper-small	0.83	0.83	✅

Polish (pl)

nEMO

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.77	0.79	✅
2	hubert-large-ls960-ft	0.40	0.39	✅
3	wav2vec2-base	0.67	0.69	✅
4	wav2vec2-large-960h	0.24	0.24	✅
5	wav2vec2-large-lv60	0.23	0.22	✅
6	wav2vec2-xls-r-1b	0.43	0.41	✅
7	wav2vec2-xls-r-300m	0.36	0.37	✅
8	wavlm-base-plus	0.73	0.74	✅
9	wavlm-large	0.66	0.66	✅
10	whisper-large	0.88	0.87	✅
11	whisper-small	0.85	0.85	✅

Portuguese (pt)

emoUERJ

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.91	0.74	✅
2	hubert-large-ls960-ft	0.59	0.51	✅
3	wav2vec2-base	0.70	0.63	✅
4	wav2vec2-large-960h	0.41	0.23	✅
5	wav2vec2-large-lv60	0.20	0.29	✅
6	wav2vec2-xls-r-1b	0.49	0.44	✅
7	wav2vec2-xls-r-300m	0.42	0.47	✅
8	wavlm-base-plus	0.88	0.74	✅
9	wavlm-large	0.88	0.76	✅
10	whisper-large	0.73	0.88	✅
11	whisper-small	0.74	0.91	✅

Quechua (qu)

Quechua-Collao-Corpus

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.81	0.80	✅
2	hubert-large-ls960-ft	0.50	0.50	✅
3	wav2vec2-base	0.78	0.76	✅
4	wav2vec2-large-960h	0.39	0.36	✅
5	wav2vec2-large-lv60	0.22	0.22	✅
6	wav2vec2-xls-r-1b	0.28	0.30	✅
7	wav2vec2-xls-r-300m	0.43	0.43	✅
8	wavlm-base-plus	0.75	0.71	✅
9	wavlm-large	0.76	0.73	✅
10	whisper-large	0.84	0.83	✅
11	whisper-small	0.82	0.82	✅

Russian (ru)

RESD

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.48	0.51	✅
2	hubert-large-ls960-ft	0.26	0.22	✅
3	wav2vec2-base	0.40	0.41	✅
4	wav2vec2-large-960h	0.23	0.24	✅
5	wav2vec2-large-lv60	0.20	0.20	✅
6	wav2vec2-xls-r-1b	0.29	0.26	✅
7	wav2vec2-xls-r-300m	0.22	0.21	✅
8	wavlm-base-plus	0.46	0.43	✅
9	wavlm-large	0.43	0.40	✅
10	whisper-large	0.48	0.46	✅
11	whisper-small	0.49	0.45	✅

Swahili (sw)

Swahili-Dataset

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.75	0.81	✅
2	hubert-large-ls960-ft	0.61	0.60	✅
3	wav2vec2-base	0.65	0.72	✅
4	wav2vec2-large-960h	0.37	0.44	✅
5	wav2vec2-large-lv60	0.41	0.35	✅
6	wav2vec2-xls-r-1b	0.35	0.45	✅
7	wav2vec2-xls-r-300m	0.35	0.33	✅
8	wavlm-base-plus	0.80	0.83	✅
9	wavlm-large	0.78	0.74	✅
10	whisper-large	0.78	0.80	✅
11	whisper-small	0.71	0.69	✅

Tamil (ta)

EmoTa

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.42	0.38	✅
2	hubert-large-ls960-ft	0.26	0.25	✅
3	wav2vec2-base	0.39	0.38	✅
4	wav2vec2-large-960h	0.30	0.24	✅
5	wav2vec2-large-lv60	0.22	0.19	✅
6	wav2vec2-xls-r-1b	0.23	0.20	✅
7	wav2vec2-xls-r-300m	0.19	0.24	✅
8	wavlm-base-plus	0.33	0.35	✅
9	wavlm-large	0.33	0.29	✅
10	whisper-large	0.40	0.41	✅
11	whisper-small	0.40	0.44	✅

Telugu (te)

Telugu-Dataset

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.48	0.53	✅
2	hubert-large-ls960-ft	0.41	0.35	✅
3	wav2vec2-base	0.46	0.44	✅
4	wav2vec2-large-960h	0.28	0.35	✅
5	wav2vec2-large-lv60	0.22	0.28	✅
6	wav2vec2-xls-r-1b	0.19	0.29	✅
7	wav2vec2-xls-r-300m	0.28	0.29	✅
8	wavlm-base-plus	0.44	0.53	✅
9	wavlm-large	0.46	0.43	✅
10	whisper-large	0.46	0.45	✅
11	whisper-small	0.50	0.52	✅

Turkish (tr)

TurEV-DB

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.66	0.68	✅
2	hubert-large-ls960-ft	0.44	0.44	✅
3	wav2vec2-base	0.43	0.48	✅
4	wav2vec2-large-960h	0.36	0.40	✅
5	wav2vec2-large-lv60	0.37	0.39	✅
6	wav2vec2-xls-r-1b	0.47	0.48	✅
7	wav2vec2-xls-r-300m	0.39	0.34	✅
8	wavlm-base-plus	0.58	0.52	✅
9	wavlm-large	0.65	0.63	✅
10	whisper-large	0.87	0.84	✅
11	whisper-small	0.88	0.84	✅

Urdu (ur)

Urdu-Dataset

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.79	0.87	✅
2	hubert-large-ls960-ft	0.35	0.43	✅
3	wav2vec2-base	0.72	0.88	✅
4	wav2vec2-large-960h	0.26	0.34	✅
5	wav2vec2-large-lv60	0.18	0.35	✅
6	wav2vec2-xls-r-1b	0.23	0.35	✅
7	wav2vec2-xls-r-300m	0.29	0.41	✅
8	wavlm-base-plus	0.72	0.76	✅
9	wavlm-large	0.64	0.50	✅
10	whisper-large	0.85	0.82	✅
11	whisper-small	0.87	0.78	✅

UrduSER

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.53	0.51	✅
2	hubert-large-ls960-ft	0.31	0.32	✅
3	wav2vec2-base	0.44	0.46	✅
4	wav2vec2-large-960h	0.23	0.24	✅
5	wav2vec2-large-lv60	0.22	0.23	✅
6	wav2vec2-xls-r-1b	0.28	0.26	✅
7	wav2vec2-xls-r-300m	0.27	0.28	✅
8	wavlm-base-plus	0.52	0.52	✅
9	wavlm-large	0.44	0.45	✅
10	whisper-large	0.60	0.60	✅
11	whisper-small	0.59	0.56	✅

Chinese (zh)

ESD

No	Model Name	Val Accuracy	Test Accuracy	Completed
1	hubert-base-ls960	0.93	0.92	✅
2	hubert-large-ls960-ft	0.75	0.73	✅
3	wav2vec2-base	0.91	0.90	✅
4	wav2vec2-large-960h	0.45	0.45	✅
5	wav2vec2-large-lv60	0.28	0.28	✅
6	wav2vec2-xls-r-1b	0.70	0.68	✅
7	wav2vec2-xls-r-300m	0.62	0.59	✅
8	wavlm-base-plus	0.91	0.90	✅
9	wavlm-large	0.90	0.91	✅
10	whisper-large	0.98	0.98	✅
11	whisper-small	0.97	0.97	✅