feat: support per-response tools in generate_reply#5310
Conversation
Allow users to specify a subset of tools for a single generate_reply call via `tools: list[str]`. Tool IDs are resolved against the agent's registered tools/toolsets. For realtime models that support per-response configuration (OpenAI), tools are passed directly on response.create. For others, session-level update_tools is used with automatic restore.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Add warning logs in Google, Phonic, Ultravox, and AWS realtime plugins when the tools param is passed to generate_reply but not supported.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Use self._rt_session.tools.flatten() instead of self.tools to capture the RT session's actual tool state, which may be a subset after filtering unsupported tools during update_tools.
|
sorry I ran out of time to play around with this today, will take a look ASAP tomorrow |
|
Can you make sure the realtime tests are passing for xAI? |
xAI realtime supports {"error": "RealtimeError(message='Invalid event received',
type='invalid_request_error', code='invalid_event', event_id='ad5486e7-1030-4849-b296-7c7331ea814a', param=None, params='1
validation error for RealtimeClientEvent\\nresponse.tools.0\\n Value error, Tool with type=\\'function\\' must have
\\'function\\' field set [type=value_error, input_value={\\'description\\': \"Called w...t\\'}, \\'type\\': \\'function\\'},
input_type=dict]\\n For further information visit https://errors.pydantic.dev/2.12/v/value_error')", "room":
"room-ca9b939b", "pid": 34502, "job_id": "AJ_Nt2K5VFGjbns", "room_id": "RM_KpLtEQjj75Xv"}I marked the |
| tool_choice (NotGivenOr[llm.ToolChoice], optional): Specifies the external tool to use when | ||
| generating the reply. If generate_reply is invoked within a function_tool, defaults to "none". | ||
| tools (NotGivenOr[list[str]], optional): List of tool IDs to make available for this response. | ||
| When set, only the specified tools can be used. Tool IDs must match registered tools on the agent. |
There was a problem hiding this comment.
| When set, only the specified tools can be used. Tool IDs must match registered tools on the agent. | |
| When set, only the specified tools can be used. Tool IDs must match registered tools on the agent. | |
| The id is typically the name of the python definition unless you have overridden it, | |
| in which case you can retrieve the id via e.g. `Assistant.{python_def_function_name}.id` |
I was initially unclear what an Id was (I think its not documented here for example https://docs.livekit.io/agents/logic/tools/definition/) so maybe something like the above could help clarify expected usage
There was a problem hiding this comment.
I tested and it works great with
await session.generate_reply(tool_choice="required", tools=[Assistant.foo.id])
Where foo is an example function
I would suggest expanding the test suite https://github.com/livekit/agents/blob/main/tests/test_realtime/test_realtime.py with an example usage ( similar to the existing test_function_tool() ) that just asserts the tool gets called
| ), | ||
| tools=( | ||
| llm.ToolContext(tools).flatten() | ||
| if per_response_tool_choice and tools |
There was a problem hiding this comment.
should we useand tools is not None here?
4 participants
Summary
tools: list[str]param togenerate_replyto scope available tools for a single responseresponse.createupdate_toolswith automatic restorefollow up of #5211