fix: extend Webset previews and preserve structured outputs

README.md

@@ -82,7 +82,7 @@ Search `output_schema` modes:
 - `{"type": "text", "description": "..."}`: return plain text in `output.content`
 - `{"type": "object", ...}`: return structured JSON in `output.content`
 
-`system_prompt` and `output_schema` are supported on every search type.
+`system_prompt` and `output_schema` are supported on every search type except `lite`.
 Search streaming is available via `stream_search(...)`, which yields OpenAI-style chat completion chunks.
 A search without `output_schema` has no text to stream. Its stream yields one chunk with the results as `citations`.
 

docs/python-sdk-specification.mdx

@@ -98,8 +98,8 @@ deep_result = exa.search(
 | moderation | Optional[bool] | If True, the search results will be moderated for safety. | None |
 | user_location | Optional[Union[str, [UserLocation](#userlocation)]] | The end user's location, as a two-letter ISO 3166-1 country code or a [UserLocation](#userlocation) dict. | None |
 | additional_queries | Optional[List[str]] | Alternative query formulations for deep search to skip automatic LLM-based query expansion. Max 5 queries. Only applicable when type is 'deep-lite', 'deep', or 'deep-reasoning'. Example: ["machine learning", "ML algorithms", "neural networks"] | None |
-| system_prompt | Optional[str] | Instructions that guide both the search process and the final returned result. Use this to prefer certain sources, emphasize novelty, avoid duplicates, or constrain the response style. Supported for all search types. | None |
-| output_schema | Optional[[DeepOutputSchema](#deepoutputschema)] | Search output schema for structured synthesis. When provided, the API returns synthesized output in ``output``. Use ``{"type": "text", "description": ...}`` for plain text output or ``{"type": "object", "properties": ..., "required": ...}`` for structured JSON. For object schemas, max nesting depth is 2 and max total properties is 10. Supported for all search types. | None |
+| system_prompt | Optional[str] | Instructions that guide both the search process and the final returned result. Use this to prefer certain sources, emphasize novelty, avoid duplicates, or constrain the response style. Supported for every search type except 'lite'. | None |
+| output_schema | Optional[[DeepOutputSchema](#deepoutputschema)] | Search output schema for structured synthesis. When provided, the API returns synthesized output in ``output``. Use ``{"type": "text", "description": ...}`` for plain text output or ``{"type": "object", "properties": ..., "required": ...}`` for structured JSON. For object schemas, max nesting depth is 2 and max total properties is 10. Supported for every search type except 'lite'. | None |
 | objective | Optional[str] | The broader goal this search serves: the task the caller is working on, beyond the query itself. When an agent picks the query as one step of a larger task, pass the goal of that step: which documents should rank first, which should be excluded, and what to pull from them. Up to 4096 characters. | None |
 | user_identifier | Optional[str] | An opaque ID you assign to the end user this request is made for, such as a UUID, used to attribute usage to that end user. Must not contain personal information such as an email address, phone number, username or name. Hash such values before sending them. 1 to 64 characters: letters, digits, ``_`` and ``-``. Retained even when your team has Zero Data Retention enabled. Does not affect results. | None |
 | betas | Optional[Sequence[str]] | Exa-Beta tokens to send with the request (e.g. ``[DYNAMIC_HIGHLIGHTS_BETA]`` for Dynamic Highlights). | None |
@@ -140,7 +140,7 @@ deep_result = exa.search(
 | url | str | The URL of the search result. |
 | id | str | The temporary ID for the document. |
 | title | Optional[str] | The title of the search result. |
-| score | Optional[float] | A number from 0 to 1 representing similarity. |
+| score | Optional[float] | Legacy field. Current search types, including the default "auto", do not return it, so it is None. Results arrive in relevance order, so keep that order instead of sorting by score. |
 | published_date | Optional[str] | An estimate of the creation date, from parsing HTML content. |
 | author | Optional[str] | The author of the content (if available). |
 | image | Optional[str] | A URL to an image associated with the content (if available). |
@@ -226,7 +226,7 @@ similar_results = exa.search(
 | url | str | The URL of the search result. |
 | id | str | The temporary ID for the document. |
 | title | Optional[str] | The title of the search result. |
-| score | Optional[float] | A number from 0 to 1 representing similarity. |
+| score | Optional[float] | Legacy field. Current search types, including the default "auto", do not return it, so it is None. Results arrive in relevance order, so keep that order instead of sorting by score. |
 | published_date | Optional[str] | An estimate of the creation date, from parsing HTML content. |
 | author | Optional[str] | The author of the content (if available). |
 | image | Optional[str] | A URL to an image associated with the content (if available). |
@@ -282,7 +282,7 @@ contents = exa.get_contents([
 | url | str | The URL of the search result. |
 | id | str | The temporary ID for the document. |
 | title | Optional[str] | The title of the search result. |
-| score | Optional[float] | A number from 0 to 1 representing similarity. |
+| score | Optional[float] | Legacy field. Current search types, including the default "auto", do not return it, so it is None. Results arrive in relevance order, so keep that order instead of sorting by score. |
 | published_date | Optional[str] | An estimate of the creation date, from parsing HTML content. |
 | author | Optional[str] | The author of the content (if available). |
 | image | Optional[str] | A URL to an image associated with the content (if available). |
@@ -1131,7 +1131,7 @@ A class representing the base fields of a search result.
 | url | str | The URL of the search result. |
 | id | str | The temporary ID for the document. |
 | title | Optional[str] | The title of the search result. |
-| score | Optional[float] | A number from 0 to 1 representing similarity. |
+| score | Optional[float] | Legacy field. Current search types, including the default "auto", do not return it, so it is None. Results arrive in relevance order, so keep that order instead of sorting by score. |
 | published_date | Optional[str] | An estimate of the creation date, from parsing HTML content. |
 | author | Optional[str] | The author of the content (if available). |
 | image | Optional[str] | A URL to an image associated with the content (if available). |
@@ -1151,7 +1151,7 @@ A class representing a search result with optional text, summary, and highlights
 | url | str | The URL of the search result. |
 | id | str | The temporary ID for the document. |
 | title | Optional[str] | The title of the search result. |
-| score | Optional[float] | A number from 0 to 1 representing similarity. |
+| score | Optional[float] | Legacy field. Current search types, including the default "auto", do not return it, so it is None. Results arrive in relevance order, so keep that order instead of sorting by score. |
 | published_date | Optional[str] | An estimate of the creation date, from parsing HTML content. |
 | author | Optional[str] | The author of the content (if available). |
 | image | Optional[str] | A URL to an image associated with the content (if available). |
@@ -1175,7 +1175,7 @@ A class representing a search result with text present.
 | url | str | The URL of the search result. |
 | id | str | The temporary ID for the document. |
 | title | Optional[str] | The title of the search result. |
-| score | Optional[float] | A number from 0 to 1 representing similarity. |
+| score | Optional[float] | Legacy field. Current search types, including the default "auto", do not return it, so it is None. Results arrive in relevance order, so keep that order instead of sorting by score. |
 | published_date | Optional[str] | An estimate of the creation date, from parsing HTML content. |
 | author | Optional[str] | The author of the content (if available). |
 | image | Optional[str] | A URL to an image associated with the content (if available). |
@@ -1196,7 +1196,7 @@ A class representing a search result with summary present.
 | url | str | The URL of the search result. |
 | id | str | The temporary ID for the document. |
 | title | Optional[str] | The title of the search result. |
-| score | Optional[float] | A number from 0 to 1 representing similarity. |
+| score | Optional[float] | Legacy field. Current search types, including the default "auto", do not return it, so it is None. Results arrive in relevance order, so keep that order instead of sorting by score. |
 | published_date | Optional[str] | An estimate of the creation date, from parsing HTML content. |
 | author | Optional[str] | The author of the content (if available). |
 | image | Optional[str] | A URL to an image associated with the content (if available). |
@@ -1217,7 +1217,7 @@ A class representing a search result with text and summary present.
 | url | str | The URL of the search result. |
 | id | str | The temporary ID for the document. |
 | title | Optional[str] | The title of the search result. |
-| score | Optional[float] | A number from 0 to 1 representing similarity. |
+| score | Optional[float] | Legacy field. Current search types, including the default "auto", do not return it, so it is None. Results arrive in relevance order, so keep that order instead of sorting by score. |
 | published_date | Optional[str] | An estimate of the creation date, from parsing HTML content. |
 | author | Optional[str] | The author of the content (if available). |
 | image | Optional[str] | A URL to an image associated with the content (if available). |
@@ -1849,7 +1849,7 @@ deprecated and kept only for backward compatibility.
 | Field | Type | Description |
 | ----- | ---- | ----------- |
 | results | Optional[List[Dict[str, Any]]] | - |
-| content | Optional[Union[str, Dict[str, Any]]] | - |
+| content | Optional[Any] | - |
 | grounding | Optional[List[[GroundingEntry](#groundingentry)]] | - |
 
 #### `SearchMonitor`

exa_py/api.py

@@ -614,7 +614,7 @@ def _validate_user_location(user_location: object) -> None:
     "additional_queries": [
         list
     ],  # Alternative query formulations for deep search variants (max 5). Only used when type is deep-lite/deep/deep-reasoning.
-    "system_prompt": [str],  # Instructions for search planning and final synthesis across all search types.
+    "system_prompt": [str],  # Instructions for search planning and final synthesis on every search type except lite.
     "output_schema": [dict],  # Search output schema: {"type":"text"} or {"type":"object", ...}
     "objective": [str],  # The broader goal the search serves, beyond the query itself (max 4096 chars).
     "user_identifier": [str],  # Opaque, non-personal identifier of the end user the request is made for.
@@ -1143,7 +1143,9 @@ class _Result:
         title (str): The title of the search result.
         url (str): The URL of the search result.
         id (str): The temporary ID for the document.
-        score (float, optional): A number from 0 to 1 representing similarity.
+        score (float, optional): Legacy field. Current search types, including the
+            default "auto", do not return it, so it is None. Results arrive in relevance
+            order, so keep that order instead of sorting by score.
         published_date (str, optional): An estimate of the creation date, from parsing HTML content.
         author (str, optional): The author of the content (if available).
         image (str, optional): A URL to an image associated with the content (if available).
@@ -2579,13 +2581,13 @@ def search(
                 Example: ["machine learning", "ML algorithms", "neural networks"]
             system_prompt (str, optional): Instructions that guide both the search process and
                 the final returned result. Use this to prefer certain sources, emphasize novelty,
-                avoid duplicates, or constrain the response style. Supported for all search types.
+                avoid duplicates, or constrain the response style. Supported for every search type except 'lite'.
             output_schema (DeepOutputSchema, optional): Search output schema for structured
                 synthesis. When provided, the API returns synthesized output in ``output``.
                 Use ``{"type": "text", "description": ...}`` for plain text output or
                 ``{"type": "object", "properties": ..., "required": ...}`` for structured JSON.
                 For object schemas, max nesting depth is 2 and max total properties is 10.
-                Supported for all search types.
+                Supported for every search type except 'lite'.
             objective (str, optional): The broader goal this search serves: the task the
                 caller is working on, beyond the query itself. When an agent picks the query
                 as one step of a larger task, pass the goal of that step: which documents
@@ -4252,13 +4254,13 @@ async def search(
                 Example: ["machine learning", "ML algorithms", "neural networks"]
             system_prompt (str, optional): Instructions that guide both the search process and
                 the final returned result. Use this to prefer certain sources, emphasize novelty,
-                avoid duplicates, or constrain the response style. Supported for all search types.
+                avoid duplicates, or constrain the response style. Supported for every search type except 'lite'.
             output_schema (DeepOutputSchema, optional): Search output schema for structured
                 synthesis. When provided, the API returns synthesized output in ``output``.
                 Use ``{"type": "text", "description": ...}`` for plain text output or
                 ``{"type": "object", "properties": ..., "required": ...}`` for structured JSON.
                 For object schemas, max nesting depth is 2 and max total properties is 10.
-                Supported for all search types.
+                Supported for every search type except 'lite'.
             objective (str, optional): The broader goal this search serves: the task the
                 caller is working on, beyond the query itself. When an agent picks the query
                 as one step of a larger task, pass the goal of that step: which documents
@@ -4586,6 +4588,7 @@ async def search_and_contents(
             data.get("resolvedSearchType"),
             data.get("autoDate"),
             context=data.get("context"),  # DEPRECATED FIELD: legacy context response.
+            output=parse_deep_search_output(data.get("output")),
             cost_dollars=cost_dollars,
             search_time=data.get("searchTime"),
             warnings=parse_search_warnings(data.get("warnings")),

exa_py/monitors/types.py

@@ -209,7 +209,8 @@ class GroundingEntry(BaseModel):
 
 class SearchMonitorRunOutput(BaseModel):
     results: Optional[List[Dict[str, Any]]] = None
-    content: Optional[Union[str, Dict[str, Any]]] = None
+    # Any JSON value. Text for a text outputSchema, otherwise JSON matching the schema.
+    content: Optional[Any] = None
     grounding: Optional[List[GroundingEntry]] = None
 
     model_config = {"populate_by_name": True}

exa_py/utils.py

@@ -99,6 +99,11 @@ def from_completion(
 
 JSONSchemaInput = Union[type[BaseModel], dict[str, Any]]
 
+# Keywords that describe a schema without constraining what it accepts.
+_ANNOTATION_KEYWORDS = frozenset(
+    {"title", "description", "examples", "default", "deprecated", "readOnly", "writeOnly", "$comment"}
+)
+
 
 class InlineJsonSchemaGenerator(GenerateJsonSchema):
     """Custom JSON schema generator that inlines all schemas without creating $defs references."""
@@ -120,29 +125,46 @@ def generate(self, schema, mode="validation"):
         return result
 
     def _inline_refs(self, obj, definitions):
-        """Recursively replace all $ref with actual definitions."""
+        """Recursively replace every resolvable ``#/$defs/`` reference with its definition.
+
+        A ``$ref`` with sibling keywords applies the definition in place beside the
+        siblings. When every sibling is an annotation (a field's description or
+        title), they are merged into the definition and win over its own. Otherwise
+        the siblings stay where they were and the definition moves into their
+        ``allOf``, which applies it in place exactly as the ``$ref`` did, so a
+        keyword that depends on what its neighbours evaluated
+        (``unevaluatedProperties``) or one the definition also sets (``required``)
+        keeps its meaning. A reference that cannot be resolved is kept, and its
+        siblings are still inlined.
+        """
         if isinstance(obj, dict):
-            if "$ref" in obj and len(obj) == 1:  # Pure ref object
-                ref_path = obj["$ref"]
-                if ref_path.startswith("#/$defs/"):
-                    def_name = ref_path[8:]  # Remove '#/$defs/'
-                    if def_name in definitions:
-                        # Replace the ref with the actual definition (recursively processed)
-                        return self._inline_refs(definitions[def_name], definitions)
-                return obj  # Return as-is if we can't resolve
-            else:
-                # Process all values in the dict
-                return {
-                    key: self._inline_refs(value, definitions)
-                    for key, value in obj.items()
-                }
+            definition = self._ref_definition(obj, definitions)
+            siblings = {
+                key: self._inline_refs(value, definitions)
+                for key, value in obj.items()
+                if definition is None or key != "$ref"
+            }
+            if definition is None:
+                return siblings
+            resolved = self._inline_refs(definition, definitions)
+            if siblings.keys() <= _ANNOTATION_KEYWORDS:
+                return {**resolved, **siblings}
+            return {**siblings, "allOf": [resolved, *siblings.get("allOf", [])]}
         elif isinstance(obj, list):
             # Process all items in the list
             return [self._inline_refs(item, definitions) for item in obj]
         else:
             # Primitive value, return as-is
             return obj
 
+    @staticmethod
+    def _ref_definition(obj, definitions):
+        """The definition ``obj["$ref"]`` points at in ``definitions``, or None."""
+        ref_path = obj.get("$ref")
+        if not isinstance(ref_path, str) or not ref_path.startswith("#/$defs/"):
+            return None
+        return definitions.get(ref_path[len("#/$defs/"):])
+
 
 def _convert_schema_input(schema_input: "JSONSchemaInput") -> dict[str, Any]:
     """Convert various schema input types to JSON Schema dict.