Agent Human Avatar Plugin

examples/avatar_agents/README.md

@@ -8,6 +8,7 @@ This directory contains examples demonstrating how to integrate visual avatars w
 
 These providers work with pre-configured avatars using unique avatar identifiers:
 
+- **[Agent Human](./agenthuman/)** - [Platform](https://agenthuman.com/) | [Integration Guide](https://docs.agenthuman.com/documentation/livekit)
 - **[Anam](./anam/)** - [Platform](https://anam.ai/) | [Integration Guide](https://docs.anam.ai/third-party-integrations/livekit)
 - **[Avatario](./avatario/)** - [Platform](https://avatario.ai/) | [Integration Guide](https://avatario.ai/docs/integrations/livekit)
 - **[Beyond Presence (Bey)](./bey/)** - [Platform](https://bey.dev/) | [Integration Guide](https://docs.bey.dev/integration/livekit)
@@ -22,6 +23,7 @@ These providers work with pre-configured avatars using unique avatar identifiers
 
 These providers allow direct image upload to create custom avatars:
 
+- **[Agent Human](./agenthuman/)** - [Platform](https://agenthuman.com/) | [Integration Guide](https://docs.agenthuman.com/documentation/livekit)
 - **[LemonSlice](./lemonslice/)** - [Platform](https://www.lemonslice.com/) | [Integration Guide](https://lemonslice.com/docs/self-managed/livekit-agent-integration)
 - **[Hedra](./hedra/)** - [Platform](https://www.hedra.com/)
 - **[BitHuman](./bithuman/)** (Cloud mode) - [Platform](https://bithuman.ai/) | [Integration Guide](https://sdk.docs.bithuman.ai/#/preview/livekit-cloud-plugin)

examples/avatar_agents/agenthuman/README.md

@@ -0,0 +1,24 @@
+# LiveKit AgentHuman Avatar Agent
+
+This example demonstrates how to create an animated avatar using [AgentHuman](https://agenthuman.com/).
+
+## Usage
+
+* Update the environment:
+
+```bash
+# AgentHuman Config
+export AGENTHUMAN_API_KEY="..."
+export AGENTHUMAN_AVATAR="..." # Avatar ID or publicly accessible image url for the avatar.
+
+# LiveKit config
+export LIVEKIT_API_KEY="..."
+export LIVEKIT_API_SECRET="..."
+export LIVEKIT_URL="..."
+```
+
+* Start the agent worker:
+
+```bash
+python examples/avatar_agents/agenthuman/agent_worker.py dev
+```

examples/avatar_agents/agenthuman/agent_worker.py

@@ -0,0 +1,40 @@
+import logging
+
+from dotenv import load_dotenv
+
+from livekit.agents import Agent, AgentServer, AgentSession, JobContext, cli, inference
+from livekit.plugins import agenthuman
+
+logger = logging.getLogger("agenthuman-avatar-example")
+logger.setLevel(logging.INFO)
+
+load_dotenv()
+
+
+server = AgentServer()
+
+
+@server.rtc_session()
+async def entrypoint(ctx: JobContext):
+    session = AgentSession(
+        stt=inference.STT("deepgram/nova-3"),
+        llm=inference.LLM("google/gemini-2.5-flash"),
+        tts=inference.TTS("cartesia/sonic-3"),
+        resume_false_interruption=False,
+    )
+
+    agenthuman_avatar = agenthuman.AvatarSession(
+        avatar="avat_01KJBCSKRWWKM9Q1ADP0QNX57D", aspect_ratio="3:4"
+    )
+    await agenthuman_avatar.start(session, room=ctx.room)
+
+    await session.start(
+        agent=Agent(instructions="Talk to me!"),
+        room=ctx.room,
+    )
+
+    session.generate_reply(instructions="say hello to the user")
+
+
+if __name__ == "__main__":
+    cli.run_app(server)

livekit-agents/livekit/agents/voice/amd/detector.py

@@ -212,7 +212,6 @@ def _resolve_classifier(
         llm: LLM | LLMModels | str | None,
         session: AgentSession,
     ) -> _AMDClassifier | None:
-
         if isinstance(llm, str):
             return _AMDClassifier(_InferenceLLM(llm))
         if isinstance(llm, _LLM):

livekit-agents/pyproject.toml

@@ -60,6 +60,7 @@ dependencies = [
 mcp = ["mcp>=1.24.0, <2"]
 codecs = ["numpy>=1.26.0"]
 images = ["pillow>=10.3.0"]
+agenthuman = ["livekit-plugins-agenthuman>=1.5.1"]
 anam = ["livekit-plugins-anam>=1.5.1"]
 anthropic = ["livekit-plugins-anthropic>=1.5.1"]
 assemblyai = ["livekit-plugins-assemblyai>=1.5.1"]

livekit-plugins/livekit-plugins-agenthuman/README.md

@@ -0,0 +1,15 @@
+# Agent Human plugin for LiveKit Agents
+
+Support for the [Agent Human](https://agenthuman.com/) virtual avatar.
+
+See the [Agent Human integration docs](https://docs.livekit.io/agents/models/avatar/plugins/agenthuman/) for more information.
+
+## Installation
+
+```bash
+pip install livekit-plugins-agenthuman
+```
+
+## Pre-requisites
+
+You'll need an API key from Agent Human. It can be set as an environment variable: `AGENTHUMAN_API_KEY`

livekit-plugins/livekit-plugins-agenthuman/livekit/plugins/agenthuman/__init__.py

@@ -0,0 +1,40 @@
+# Copyright 2023 LiveKit, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""AgentHuman virtual avatar plugin for LiveKit Agents
+
+See https://docs.livekit.io/agents/models/avatar/plugins/agenthuman/ for more information.
+"""
+
+from livekit.agents import Plugin
+
+from .api import AgentHumanException
+from .avatar import AvatarSession
+from .log import logger
+from .version import __version__
+
+__all__ = [
+    "AgentHumanException",
+    "AgentHumanPlugin",
+    "AvatarSession",
+    "__version__",
+]
+
+
+class AgentHumanPlugin(Plugin):
+    def __init__(self) -> None:
+        super().__init__(__name__, __version__, __package__, logger)
+
+
+Plugin.register_plugin(AgentHumanPlugin())

livekit-plugins/livekit-plugins-agenthuman/livekit/plugins/agenthuman/api.py

@@ -0,0 +1,141 @@
+import asyncio
+import os
+from typing import Any, cast
+
+import aiohttp
+
+from livekit.agents import (
+    DEFAULT_API_CONNECT_OPTIONS,
+    NOT_GIVEN,
+    APIConnectionError,
+    APIConnectOptions,
+    APIStatusError,
+    NotGivenOr,
+    utils,
+)
+
+from .log import logger
+
+
+class AgentHumanException(Exception):
+    """Exception for AgentHuman errors"""
+
+
+DEFAULT_AVATAR_ID = "avat_01KMZJ4WE5QCD9G3EK21QW8NJK"
+
+
+class AgentHumanAPI:
+    def __init__(
+        self,
+        api_key: NotGivenOr[str] = NOT_GIVEN,
+        *,
+        conn_options: APIConnectOptions = DEFAULT_API_CONNECT_OPTIONS,
+        session: aiohttp.ClientSession | None = None,
+    ) -> None:
+        agenthuman_api_key = api_key or os.getenv("AGENTHUMAN_API_KEY")
+        if agenthuman_api_key is None:
+            raise AgentHumanException("AGENTHUMAN_API_KEY must be set")
+        self._api_key = agenthuman_api_key
+
+        self._api_url = "https://api.agenthuman.com/v1"
+        self._conn_options = conn_options
+        if session is not None:
+            self._session = session
+            self._owns_session = False
+        else:
+            self._session = aiohttp.ClientSession()
+            self._owns_session = True
+
+    async def aclose(self) -> None:
+        if self._owns_session:
+            await self._session.close()
+
+    async def create_session(
+        self,
+        *,
+        avatar: NotGivenOr[str] = NOT_GIVEN,
+        aspect_ratio: NotGivenOr[str] = NOT_GIVEN,
+        livekit_room: NotGivenOr[dict[str, Any]] = NOT_GIVEN,
+        extra_payload: NotGivenOr[dict[str, Any]] = NOT_GIVEN,
+    ) -> str:
+        """
+        Create a session with the AgentHuman API.
+
+        Args:
+            avatar: Avatar ID to use for the session
+            aspect_ratio: Aspect ratio to use for the session
+            livekit_room: LiveKit room configuration dictionary
+            extra_payload: Additional payload to merge into the request
+
+        Returns:
+            Session ID string
+
+        Raises:
+            AgentHumanException: If avatar is not provided
+            APIConnectionError: If the request fails
+        """
+        avatar = avatar or (os.getenv("AGENTHUMAN_AVATAR") or DEFAULT_AVATAR_ID)
+
+        room_url = livekit_room.get("livekit_ws_url") if utils.is_given(livekit_room) else None
+        room_token = (
+            livekit_room.get("livekit_room_token") if utils.is_given(livekit_room) else None
+        )
+        payload = {
+            "avatar": avatar,
+            "room": {"platform": "livekit", "url": room_url, "token": room_token},
+        }
+        if utils.is_given(aspect_ratio):
+            payload["aspect_ratio"] = aspect_ratio
+        if utils.is_given(extra_payload):
+            payload.update(extra_payload)
+
+        last_exc: Exception | None = None
+        for i in range(self._conn_options.max_retry):
+            try:
+                async with self._session.post(
+                    f"{self._api_url}/sessions",
+                    headers={
+                        "Content-Type": "application/json",
+                        "x-api-key": self._api_key,
+                    },
+                    json=payload,
+                    timeout=aiohttp.ClientTimeout(sock_connect=self._conn_options.timeout),
+                ) as response:
+                    if not response.ok:
+                        text = await response.text()
+                        raise APIStatusError(
+                            "Server returned an error", status_code=response.status, body=text
+                        )
+                    session_data = await response.json()
+                    try:
+                        return cast(str, session_data["session"]["session_id"])
+                    except (KeyError, TypeError) as e:
+                        raise AgentHumanException(
+                            f"Unexpected API response structure: {session_data}"
+                        ) from e
+            except AgentHumanException:
+                raise
+            except APIStatusError as e:
+                last_exc = e
+                if not e.retryable:
+                    raise
+                logger.warning(
+                    "[agenthuman] failed to call agenthuman api", extra={"error": str(e)}
+                )
+            except Exception as e:
+                last_exc = e
+                if isinstance(e, APIConnectionError):
+                    logger.warning(
+                        "[agenthuman] failed to call agenthuman api", extra={"error": str(e)}
+                    )
+                else:
+                    logger.exception("[agenthuman] failed to call agenthuman api")
+
+            if i < self._conn_options.max_retry - 1:
+                await asyncio.sleep(self._conn_options.retry_interval)
+
+        if isinstance(last_exc, APIStatusError):
+            raise last_exc
+        raise APIConnectionError(
+            "Failed to create AgentHuman session after all retries"
+        ) from last_exc