Emotion2Vec Speech Emotion API

Upload audio to FastAPI and receive the highest-scoring label from a pinned ModelScope emotion2vec pipeline.

Experimental inference wrapper with local execution after the model is downloaded.

What is it? · Quick start · API · Architecture · Runtime notes · Limitations

What is it?

This repository is a small FastAPI wrapper for developers experimenting with utterance-level speech-emotion inference. At process startup, app.py creates a ModelScope pipeline for iic/emotion2vec_base_finetuned at revision v2.0.4.

Requests run on the host machine: uploaded bytes are passed to the downloaded model, NumPy selects the largest score, and a fixed label map is returned as JSON. ModelScope access is needed when the model is not already cached; the repository does not call a separately hosted inference API.

Quick start

Create an isolated Python environment. The first install and first model load can be large because the dependency set includes FunASR, ModelScope, TorchAudio, and dataset tooling.

git clone https://github.com/jayanth-mkv/emotion2vec-speech-emotion-detection-api.git
cd emotion2vec-speech-emotion-detection-api
python -m venv .venv
python -m pip install -r requirements.txt
uvicorn app:app --reload

Open http://localhost:8000/docs for FastAPI's generated interface.

API

POST /emotion_recognition

Send one multipart field named audio_file:

curl -X POST \
  -F "audio_file=@/path/to/sample.wav" \
  http://localhost:8000/emotion_recognition

Response field	Meaning
emotion	Label at the index of the largest score.
confidence	The pipeline score at that index; the repository does not document calibration.

The hard-coded index order is:

angry, disgust, fear, happy, neutral, other, sad, surprised, unknown

That order must remain compatible with the pinned model revision. The API does not currently expose the full score vector.

Architecture

Runtime notes

• The model pipeline is created while app.py imports, so server readiness waits for model initialization.
• Inference writes to ./outputs through the pipeline's output_dir argument.
• The endpoint uses utterance granularity and disables embedding extraction.
• vercel.json expresses a planned Vercel Python deployment, but no deployed URL or serverless verification is included.

Limitations

• There is no audio MIME, codec, duration, sample-rate, channel-count, or upload-size validation.
• Model download size, memory use, latency, and hardware requirements are not measured in this repository.
• The label mapping is maintained manually rather than read from model metadata.
• Requests have no authentication, rate limit, queue, timeout, health endpoint, or structured model-error handling.
• The score is reported as provided by the model and should not be treated as a calibrated probability without separate evaluation.
• There are no tests or evaluation artifacts proving accuracy on IEMOCAP or any other dataset in this repository.
• No license file is included, so reuse terms are not currently stated.