Repository navigation
fix: Azure Blob (private container) image support: authenticated vision downloads + UI display - #13411
fix: Azure Blob (private container) image support: authenticated vision downloads + UI display#13411rahepler2 wants to merge 8 commits into
Conversation
|
Addresses: Discussion:
Closes: |
|
@danny-avila when you review this, I did see a PR for a similar approach. (#13245). The core difference is, mine authenticates Blob reads with the Container App's managed identity and needs only Storage Blob Data Reader, and not Storage Blob Delegator, account key, and signed URLs. They don't need to be mutually exclusive depending on which path you'd prefer. Even if the SAS is preferred the fall back for soveriegn clouds that cannot allow delegation could use the path I have chosen. |
|
Thanks for picking this up, @rahepler2 — agreed these aren't mutually exclusive, and I think they slot together cleanly. Sketching the two access models so @danny-avila has the full picture:
The one genuinely shared piece is the server-side Two small notes: #13245 already implements the Managed Identity path (so it also covers #10784), and it handles the sovereign-cloud endpoint via a full-URL |
|
@JorgeCosta87 Hey, thanks for writing up the update! I did realize in testing there were a few more routes that needed to be covered by this same azure blob MI and SAS approach. Look at the latest commit to this branch I just pushed, but the proxy for agent, tool, and generated images were also lacking the necessary support. So for exampled, if I call my MCP tool to get an image back of something, that would use the route to get it into blob storage correctly, but wasn't able to display it. Updated those accordingly, but will need it on your side (if you haven't already done so). |
|
Pushed a fix for the failing test:ci job. The two failures (ArtifactRouting, OpenAIImageGen) were because those suites render the converted components in isolation without the app providers, and the new ImageFile component uses Recoil + React Query. I mocked ImageFile in both suites with the same leaf-mock pattern they already use for Image. The assertions are unchanged. Full client suite passes locally. |
|
@danny-avila Wanted to see about review for this. |
|
removed the added env on mine and adjusted to use AZURE_STORAGE_BLOB_ENDPOINT |
26c974d to
6dd58e5
Compare
|
Hey @danny-avila, just wanted to bump this and see about review. We are looking to move towards larger testing and would need this in place. |
f2ed405 to
9624b57
Compare
|
IMHO, this is currently a huge blocker for Azure users who want/need to enable secure file upload and storage in the private containers. Any timeline when this will be moved forward? Thanks. |
6e3aae5 to
4d593ee
Compare
Waiting for review and approval from maintainers on this |
4d593ee to
c3b390e
Compare
|
👋 Thanks for the contribution! LibreChat merges all changes into Nothing is needed from you; your commits, reviews and discussion are unchanged. If the diff now shows files you did not touch, rebase onto git remote add upstream https://github.com/LibreChat-AI/LibreChat.git
git fetch upstream dev
git rebase upstream/dev
git push --force-with-leaseMaintainers: apply the |
Replace anonymous HTTP blob fetches in getAzureFileStream with the SDK-authenticated container client so private/non-public containers (default in regulated environments, only option in GCC High) work for vision payloads instead of returning 403. Fail closed in encodeAndFormat when a blob download fails rather than silently falling back to a URL the remote LLM cannot fetch, and parameterize the storage endpoint suffix for sovereign clouds via AZURE_STORAGE_ENDPOINT_SUFFIX.
Chat images from object-storage sources that require server-side auth (e.g. private Azure Blob) cannot be loaded by an <img> tag, which fires an unauthenticated request the browser cannot sign. Such images previously rendered only when the container allowed anonymous public access. Add a reusable ImageFile component that, for these sources, fetches the bytes through the existing authenticated /files/download proxy (which streams using the server's credentials) and renders them from a local blob: URL. All other sources continue to render their URL directly. Used by both the message content (Part) and attachment (Files) render paths.
Limit the encodeAndFormat fail-closed throw to azure_blob so S3, Firebase, and CloudFront keep falling back to their fetchable URLs on a download error. Render proxy-eligible images (private Azure Blob) from their direct URL first and fetch through the authenticated download proxy only when the direct load fails, so public Azure containers continue to load images directly instead of routing every image through the backend.
Route the tool-output (LogContent), attachment (Attachment), and image-generation (OpenAIImageGen) render paths through ImageFile so images from auth-required sources (private Azure Blob) load via the authenticated download proxy instead of an anonymous <img> request that a private container rejects. Add an optional args passthrough to ImageFile for the image-generation dialog metadata.
Drop AZURE_STORAGE_ENDPOINT_SUFFIX (env var, azure.ts handling, .env.example docs, and its test) to consolidate on LibreChat-AI#13245's AZURE_STORAGE_BLOB_ENDPOINT full-URL override, per review discussion. The backend authenticated download and the frontend ImageFile display proxy are unchanged.
SteerPart (new on dev) rendered attached images with Image directly, so private Azure Blob images on steer messages failed to load. Render them through ImageFile, keeping SteerPart's size and alt-text fallbacks. Wrap Files and Part image tests in Recoil/React Query providers, which ImageFile needs, and apply prettier/import sorting to touched files.
c3b390e to
2bf72df
Compare
|
@danny-avila Wanted to check back in on this. I'm seeing more issues pop up around this and more PR's coming in. Is there something your'd like me to do to organize things further between Jorge and my original PR? |
Summary
Makes Azure Blob storage work end-to-end with a private container
(
AZURE_STORAGE_PUBLIC_ACCESS=false). Previously, images only worked if thecontainer was left publicly readable, which isn't acceptable for enterprise /
regulated / sovereign-cloud deployments.
With a private container, two things were broken:
https://<account>.blob.core.windows.net/...URL that remote providers can'tfetch.
getAzureFileStreamused an anonymous HTTP GET, andencodeAndFormatsilently fell back to that unfetchable URL.
<img>pointed at the same private URL; an<img>tag fires an unauthenticated request (it can't attach the server's Managed
Identity or the LibreChat JWT), so the browser got a 403.
Changes
getAzureFileStreamnow downloads via the authenticatedAzure SDK (Managed Identity /
DefaultAzureCredential), matching the existingdeleteFileFromAzurepattern.encodeAndFormatnow fails closed forazure_blob(throws a clear error) instead of falling back to a private URL.The fail-closed is scoped to
azure_blobonly, so S3 / Firebase / CloudFrontkeep their existing fallback to their fetchable URLs on a download error.
ImageFilecomponent renders the image fromits direct URL first and, only if that load fails, fetches the bytes through
the existing authenticated
/files/downloadproxy (which streams using theserver's credentials) and renders them from a local
blob:URL. Public Azurecontainers continue to load directly; private containers fall back to the proxy.
Other sources render their URL directly, unchanged.
AZURE_STORAGE_ENDPOINT_SUFFIX(defaults tocore.windows.net); documented in.env.example.No SAS tokens and no new RBAC role — the existing
Storage Blob Data Readeronthe app's identity covers both the vision and display paths. Public-container
behavior is unchanged.
Change Type
Testing
Configure LibreChat with Azure Blob storage using Managed Identity and a
private container (
AZURE_STORAGE_PUBLIC_ACCESS=false). Grant the app'sidentity
Storage Blob Data Readeron the storage account.contains a
*.blob.core.*URL.data:image/...;base64,...and themodel responds correctly.
authenticated
/files/downloadproxy as ablob:URL).AZURE_STORAGE_ENDPOINT_SUFFIX=core.usgovcloudapi.netand confirm the blob service URL uses that suffix.
Automated tests added/updated:
api/.../Azure/__tests__/crud.test.js— SDK download path parsing & failure modes.api/.../images/__tests__/encode.test.js—encodeAndFormatfails closed forazure_blob.packages/api/src/cdn/__tests__/azure.test.ts— endpoint suffix default + override.client/.../Content/__tests__/ImageFile.test.tsx— direct render, proxy-on-error, local preview.client/.../Files/__tests__/previewRefetchInterval.spec.ts—isProxyImageSource.npm run test:client(2279 tests) passes; backend tests for the changed filespass. Note: the unrelated
server/routes/agents/__tests__/responses.spec.jsintegration suite fails locally without LLM provider credentials and is not
affected by this change.
Test Configuration:
DefaultAzureCredential), privatecontainer, Azure Container Apps. Node 20.
Checklist