Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Unstructured API key - get yours from the Transform get-started page:
# https://transform.unstructured.io/get-started
UNSTRUCTURED_API_KEY=

# AWS credentials for Amazon Bedrock model access
AWS_ACCESS_KEY_ID=your_access_key_id
AWS_SECRET_ACCESS_KEY=your_secret_access_key
AWS_REGION=us-west-2
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Federating Transform MCP through Amazon Bedrock AgentCore Gateway

This guide covers an alternative deployment pattern for the sample in this folder: instead of an agent connecting directly to `https://mcp.transform.unstructured.io`, you federate Transform MCP as a **target** behind an [Amazon Bedrock AgentCore Gateway](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway-target-MCPservers.html), and point the agent at the Gateway's unified MCP endpoint instead.

## What AgentCore Gateway is

AgentCore Gateway is a managed service that turns existing APIs, Lambda functions, and other MCP servers into a single MCP endpoint for your agents. It handles:

- **Inbound auth** to the Gateway itself (who is allowed to call the Gateway's MCP endpoint).
- **Outbound auth** to each federated target (how the Gateway authenticates to Transform MCP, a Lambda function, an OpenAPI backend, etc.), via **credential providers** it manages on your behalf.
- **Tool discovery and indexing**: the Gateway calls each target's `tools/list` and indexes the results into one searchable, unified tool catalog.

## Why federate Transform MCP through a Gateway

For a single-agent sample like the one in this folder, connecting directly to Transform MCP is simplest. Federating through AgentCore Gateway is worth the extra setup when:

- You want **one managed endpoint** in front of several MCP servers (Transform MCP plus internal tools, other vendor MCP servers, etc.), instead of wiring every agent to every server individually.
- You want **credential management centralized** in AWS rather than distributing an `UNSTRUCTURED_API_KEY` (or OAuth client) to every agent runtime.
- You want a **unified tool catalog** with search/discovery across many federated targets, rather than agents each calling `list_tools_sync()` against a fixed set of servers.

## Adding Transform MCP as a Gateway target

Use `bedrock-agentcore-control`'s `create_gateway_target` API with an `mcp` target configuration pointing at Transform MCP's endpoint, and an `OAUTH` credential provider for outbound auth:

```python
import boto3

client = boto3.client("bedrock-agentcore-control", region_name="us-west-2")

response = client.create_gateway_target(
gatewayIdentifier="<your-gateway-id>",
name="unstructured-transform",
description="Unstructured Transform document-processing MCP server",
targetConfiguration={
"mcp": {
"mcpServer": {
"endpoint": "https://mcp.transform.unstructured.io",
}
}
},
credentialProviderConfigurations=[
{
"credentialProviderType": "OAUTH",
"credentialProvider": {
"oauthCredentialProvider": {
"providerArn": "<your-oauth-credential-provider-arn>",
"scopes": [],
}
},
}
],
)
```

Transform MCP's browser OAuth/OIDC flow is a 3-legged, authorization-code grant. After `create_gateway_target` returns, the target sits in `CREATE_PENDING_AUTH` until an admin completes the authorization URL for that credential provider (a one-time, human-in-the-loop step). Once authorized, the Gateway can call the target's `tools/list` and indexes Transform's tools (`start_transform_job`, `check_job_status`, `get_job_results`, `request_file_upload_url`) into the Gateway's unified catalog.

See the AWS docs for the full target configuration schema and current field names:

- [Gateway targets for MCP servers](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway-target-MCPservers.html)
- [Gateway target API configuration reference](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway-add-target-api-target-config.html)

## Pointing the Strands agent at the Gateway instead

Once the target is authorized and active, update the agent from this sample to connect to the Gateway's MCP endpoint rather than Transform MCP directly. The client code is the same shape (`streamablehttp_client` + `MCPClient`); only the URL and the auth header change, since the Gateway now mediates its own inbound auth:

```python
import os
from mcp.client.streamable_http import streamablehttp_client
from strands.tools.mcp import MCPClient
from strands import Agent

mcp_client = MCPClient(lambda: streamablehttp_client(
url="<your-agentcore-gateway-mcp-endpoint>",
headers={"Authorization": f"Bearer {os.environ['GATEWAY_ACCESS_TOKEN']}"},
))

with mcp_client:
tools = mcp_client.list_tools_sync()
agent = Agent(model="us.anthropic.claude-sonnet-4-5-20250929-v1:0", tools=tools)
response = agent("Parse and chunk this document: <public PDF URL>")
```

`tools` returned here may include Transform's tools alongside tools from any other targets federated on the same Gateway. The agent code itself doesn't need to know which target a given tool came from.
112 changes: 112 additions & 0 deletions python/03-integrate/protocols/unstructured-transform-mcp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Unstructured Transform MCP

A Strands agent that connects to the hosted Unstructured Transform MCP server to parse, chunk, and enrich documents through an asynchronous processing pipeline.

## Overview

[Unstructured Transform](https://docs.unstructured.io/transform/overview) is a hosted, remote MCP server that exposes Unstructured's document-processing pipeline as MCP tools, parsing PDFs, spreadsheets, scans, and many file types with tables and layout intact. Instead of running a local document-parsing binary, your agent calls a hosted service over streamable-http and gets back parsed text, chunks, tables, image descriptions, or embeddings, depending on which pipeline stages you request.

### Sample Details

| Information | Details |
|------------------------|-------------------------------------------------------------|
| **Agent Architecture** | Single-agent |
| **Native Tools** | None |
| **Custom Tools** | None |
| **MCP Servers** | [Unstructured Transform MCP](https://mcp.transform.unstructured.io) |
| **Use Case Vertical** | Document processing / RAG |
| **Complexity** | Intermediate |
| **Model Provider** | Amazon Bedrock |
| **SDK Used** | Strands Agents SDK |

### Architecture

```mermaid
sequenceDiagram
participant User
participant Agent as Strands Agent
participant Transform as Unstructured Transform MCP<br/>(streamable-http)

User->>Agent: "Parse and chunk this document: <PDF URL>"
Agent->>Transform: start_transform_job(file_refs, stages)
Transform-->>Agent: job_id
loop Poll until complete
Agent->>Transform: check_job_status(job_id)
Transform-->>Agent: status
end
Agent->>Transform: get_job_results(job_id, output_format)
Transform-->>Agent: rendered output (md/json/html/text)
Agent-->>User: Summary of parsed & chunked document
```

The agent connects to Transform MCP over `streamable-http`, authenticating with an Unstructured API key passed as a bearer token. It discovers the server's tools at runtime via `list_tools_sync()`, then drives the async pipeline: submit a job with `start_transform_job`, poll `check_job_status`, and fetch the rendered output with `get_job_results`.

### Key Features

- **Hosted, remote MCP server**: no local binaries, containers, or native dependencies (e.g. LibreOffice, poppler) to install, just a URL and an API key.
- **Async job pipeline**: `start_transform_job` returns a `job_id` immediately; the agent polls `check_job_status` and fetches results with `get_job_results` once complete, matching how a production integration would handle longer-running documents.
- **Configurable pipeline stages**: partition (`auto` / `fast` / `hi_res` / `vlm`), enrich (image descriptions, table-to-HTML, NER, generative OCR), chunk, and embed stages can be composed per request via the `stages` argument.
- **Two auth paths**: browser OAuth/OIDC for interactive clients, or an API-key bearer token for headless frameworks like this one.

## Prerequisites

- Python **3.10+**
- [uv](https://docs.astral.sh/uv/getting-started/installation/) for dependency management
- An Unstructured API key from the [Transform get-started page](https://transform.unstructured.io/get-started) (free tier includes 15,000 pages a month)
- AWS CLI configured with credentials that have [Amazon Bedrock model access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access-modify.html) enabled for Claude Sonnet 4.5

## Setup

1. **Configure environment variables:**
```bash
cp .env.example .env
# Edit .env and set UNSTRUCTURED_API_KEY and your AWS credentials
```

2. **Install dependencies:**
```bash
uv sync
```

## Usage

**Run the sample:**
```bash
uv run main.py
```

The agent connects to Transform MCP, lists the available tools, then submits a small public sample PDF for parsing and chunking, polls the job until it completes, and prints a summary of the results.

If `UNSTRUCTURED_API_KEY` is not set, the script exits immediately with a clear error message pointing to the Transform get-started page.

## Comparison to AWS Labs Document Loader

[`awslabs.document-loader-mcp-server`](https://github.com/awslabs/mcp/tree/main/src/document-loader-mcp-server) is a local, stdio-only MCP server with three synchronous, single-shot tools (`read_document`, `read_image`, `extract_slides_as_images`). It depends on native binaries (LibreOffice, poppler-utils) installed on the host, and has no hosted/remote transport, no OAuth story, and no job/status model.

Unstructured Transform MCP is a hosted alternative for the same broad task (getting document content into an agent) with a different capability profile:

| | AWS Labs Document Loader | Unstructured Transform MCP |
|---|---|---|
| Transport | stdio (local process) | streamable-http (hosted) |
| Execution model | Synchronous, single-shot | Async job (submit → poll → fetch) |
| Native dependencies | LibreOffice, poppler-utils | None (fully hosted) |
| Auth | None | OAuth/OIDC or API-key bearer token |
| Pipeline depth | Basic text/image extraction | Configurable partition (incl. `hi_res`/VLM), enrichment (table/image descriptions, NER, OCR), chunking, and embeddings |

Choose whichever fits your deployment: Document Loader for a fully local, dependency-managed extraction step; Transform MCP when you want a hosted pipeline with richer partitioning, enrichment, chunking, and embedding stages and don't want to manage native binaries yourself.

## AgentCore Gateway

If you want to expose Transform MCP's tools alongside other MCP servers behind a single managed endpoint (with centralized auth and a unified tool catalog), see [AGENTCORE_GATEWAY.md](./AGENTCORE_GATEWAY.md) for a guide on federating Transform MCP as an Amazon Bedrock AgentCore Gateway target.

---

## Disclaimer

This sample is provided for educational and demonstration purposes only. It is not intended for production use without further development, testing, and hardening.

For production deployments, consider:
- Implementing appropriate content filtering and safety measures
- Following security best practices for your deployment environment
- Conducting thorough testing and validation
- Reviewing and adjusting configurations for your specific requirements
117 changes: 117 additions & 0 deletions python/03-integrate/protocols/unstructured-transform-mcp/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
"""Unstructured Transform MCP + Strands Agent sample.

This sample connects a Strands Agent to the hosted Unstructured Transform MCP
server (https://mcp.transform.unstructured.io) over the streamable-http
transport and asks the agent to parse and chunk a public sample PDF.

Transform MCP exposes an *asynchronous* document-processing pipeline through
four tools:

start_transform_job(file_refs, stages) -> job_id
check_job_status(job_id) -> status
get_job_results(job_id, output_format) -> rendered output
request_file_upload_url() -> presigned URL for local files

Because the pipeline is async, this sample gives the agent explicit
instructions to submit the job, poll for completion, and then fetch the
rendered results - rather than relying on a single free-form prompt. This
makes the async job lifecycle visible in the console output.

Prerequisites:
- An Unstructured API key (see the Transform get-started page at
https://transform.unstructured.io/get-started) exported as UNSTRUCTURED_API_KEY.
- AWS credentials with Amazon Bedrock model access configured in your
environment (see .env.example).

Docs: https://docs.unstructured.io/transform/overview

Usage:
uv run main.py
"""

import os
import sys

from mcp.client.streamable_http import streamablehttp_client
from strands import Agent
from strands.tools.mcp import MCPClient

# Hosted Unstructured Transform MCP server (streamable-http transport).
TRANSFORM_MCP_URL = "https://mcp.transform.unstructured.io"

# Bedrock model used to drive the agent. Requires model access to be enabled
# in your AWS account/region; see the README Prerequisites section.
MODEL_ID = "us.anthropic.claude-sonnet-4-5-20250929-v1:0"

# A small, stable, publicly-reachable PDF used purely to demonstrate the
# pipeline end-to-end. Swap this for any https:// URL, or use
# request_file_upload_url() first if you want to process a local file.
SAMPLE_PDF_URL = "https://arxiv.org/pdf/1706.03762"


def build_mcp_client(api_key: str) -> MCPClient:
"""Create an MCPClient wired up to the hosted Transform MCP server.

Transform MCP supports two auth modes: browser OAuth/OIDC, and an
API-key mode for headless frameworks like this one, where the key is
passed as a bearer token in the Authorization header.
"""

def create_transport():
return streamablehttp_client(
url=TRANSFORM_MCP_URL,
headers={"Authorization": f"Bearer {api_key}"},
)

return MCPClient(create_transport)


def main() -> None:
api_key = os.environ.get("UNSTRUCTURED_API_KEY")
if not api_key:
print(
"ERROR: UNSTRUCTURED_API_KEY is not set.\n"
"Get a key from the Transform get-started page (https://transform.unstructured.io/get-started) "
"and export it, e.g.:\n\n"
" export UNSTRUCTURED_API_KEY=<your-key>\n",
file=sys.stderr,
)
sys.exit(1)

mcp_client = build_mcp_client(api_key)

with mcp_client:
# Discover the tools Transform MCP exposes (start_transform_job,
# check_job_status, get_job_results,
# request_file_upload_url).
tools = mcp_client.list_tools_sync()
print(f"Connected to Transform MCP. Discovered {len(tools)} tool(s):")
for tool in tools:
print(f" - {tool.tool_name}")
print()

agent = Agent(model=MODEL_ID, tools=tools)

# Give the agent explicit steps so the async job lifecycle
# (submit -> poll -> fetch) is exercised deterministically, rather
# than leaving the whole flow to the model's discretion.
prompt = f"""
Process this document using the Transform tools: {SAMPLE_PDF_URL}

Follow these steps exactly:
1. Call start_transform_job with file_refs=["{SAMPLE_PDF_URL}"] and stages
configured for a partition (strategy "auto") followed by a chunk stage
with default settings. This returns a job_id.
2. Call check_job_status with that job_id repeatedly (waiting a few
seconds between calls) until the status indicates the job is complete.
3. Call get_job_results with the job_id and output_format="md".
4. Summarize the first two chunks of the returned markdown in 2-3 sentences.
"""

response = agent(prompt)
print("\n--- Agent response ---")
print(response)


if __name__ == "__main__":
main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[project]
name = "unstructured-transform-mcp"
version = "0.1.0"
description = "Strands Agent sample using the hosted Unstructured Transform MCP server for document processing"
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
"strands-agents>=0.1.0",
"strands-agents-tools>=0.1.0",
"mcp>=1.9.0",
]
Loading