Repository navigation
Feat/interaction logs - #16859
Closed
Vidminas wants to merge 35 commits into
Closed
Feat/interaction logs#16859Vidminas wants to merge 35 commits into
Vidminas wants to merge 35 commits into
Conversation
…r scoping Replace the rag_api/pgvector vector store with an AWS Bedrock Knowledge Base (ministack-provisioned locally): files are written to the KB's S3 data source with a .metadata.json sidecar, ingestion jobs are debounced, and file_search and attached-file context retrieve via bedrock-agent-runtime Retrieve. User separation: retrieveForFiles scopes a user's own uploads by file_id and user_id, agent files by file_id alone, re-checks every result against the entitled set, and returns nothing without entitled files. Unscoped retrieves are refused, and object-key filenames are sanitised. Also unify the ragConfigured checks behind isBedrockKbConfigured(), blank RAG_API_URL for the app so text extraction uses the native parser, and rewrite the stale rag_api fileSearch tests plus add scope specs.
This reverts commit d091d5d.
This reverts commit def82c0.
This reverts commit 0b2e49a.
… per-user scoping" This reverts commit 9260524.
The revert left RAG_API_URL de-indented to the top of `services`, so the override declared it as a service instead of passing it to the API, and rag_api/vectordb stayed parked behind the `legacy-rag` profile even though the API depends on rag_api again. Put RAG_API_URL back in the API environment and drop the profile gates so both services start with the stack.
DocumentDB requires TLS verified against the AWS CA bundle, which Mongoose only picks up through `tlsCAFile`. Pass MONGO_TLS_CA_FILE through to the connection options when it is set; local MongoDB leaves it empty and connects as before. In the compose override the API now takes MONGO_URI from .env instead of a blank value, forwards MONGO_TLS_CA_FILE, and mounts ./certs/documentdb read-only for the bundle (the bundle itself is gitignored). The local mongodb service becomes an opt-in single-node replica set behind the `mongodb-compat` profile, so the compatibility suites can exercise transactions. It is a baseline only, not a DocumentDB emulator.
Cover the network path (VPN or tunnel), the CA bundle, the MONGO_URI parameters each connection type needs, and the `mongodb-compat` baseline profile.
Real Bedrock inference needs real credentials, while the rest of the local stack keeps talking to ministack with static test keys. Mount the host's AWS config directory (AWS_PROFILE_DIR) read-only into the API container and pass BEDROCK_AWS_PROFILE and BEDROCK_AWS_DEFAULT_REGION through. The global AWS_ENDPOINT_URL points every SDK client at ministack, including the ones that resolve a profile. Pin the SSO, SSO OIDC and STS endpoints to real AWS so profile and SSO resolution still work.
Both directories are runtime volumes. Without the exclusion `COPY . .` sent several gigabytes of model blobs to the daemon on every build and baked their private keys into the image.
Offer Amazon Nova Lite v1 alongside Claude Sonnet 5. eu-west-2 serves it on-demand only, so it is listed by its foundation-model ID rather than an `eu.` inference profile.
Adds an `analytics` database of read-only views over the HyperDX and Langfuse tables: UI actions and page views, LLM traces, model calls (model, tokens, cost, latency) and feedback scores, plus a `timeline` view that puts all of them on one per-user timeline in the column layout HyperDX expects of a log source. The views leave out every prompt, response, metadata field and feedback comment. HyperDX's ClickHouse user may now SELECT from these views but still cannot read Langfuse's tables. A one-shot clickhouse-analytics-init Compose service creates the views once Langfuse and the collector have created their tables. The SQL drops and re-creates the views instead of using CREATE OR REPLACE, which fails on the EFS volume ClickHouse uses on AWS.
Adds a "Usage timeline" HyperDX source over the analytics.timeline view, a saved search of the same name, and an "LLM usage" dashboard: model calls, tokens, cost, response time, feedback and failed calls over time, usage per user, UI actions and the event timeline, with User, Model and Event source filters. HyperDX applies DEFAULT_SOURCES only to a team that has no sources yet, and its own dashboard provisioner needs per-installation source ids, so the files name their source and connection instead. provision.js runs on the HyperDX image, resolves those names per team with HyperDX's own models and validation, and upserts everything by name. Locally it runs as the one-shot hyperdx-provision Compose service.
Langfuse's LANGFUSE_INIT_* bootstrap only creates the admin user, so a rotated LANGFUSE_INIT_USER_PASSWORD never reached an existing admin and locked them out. admin.js runs on the Langfuse image as a one-off task and sets the admin's password from the environment, hashing it with the same bcryptjs library and cost Langfuse uses. It writes only when the password differs and changes nothing before the admin exists.
The sandbox is an AWS channel-program account, where Bedrock refuses Anthropic models. Adds the models it does serve on demand in eu-west-2, so requests stay in London: gpt-oss-120b, Qwen3 235B, Kimi K2.5, DeepSeek V3.2, Devstral 2, GLM-5 and Nemotron Super 3. Each was checked with streaming and a tool call round trip.
OpenTofu hashes librechat.*.yaml and the files under otel/ into image tags and one-off task triggers. A CRLF checkout on one machine would change those hashes, so every apply from it would rebuild images and re-run tasks that an apply from another machine had just settled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The OpenTofu stack moved to terraform/ at the repo root, and its files were regrouped by service: the analytics views now live in clickhouse.tf. Comments only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A HyperDX source links to others by id (traceSourceId, logSourceId, metricSourceId, sessionSourceId), and ids differ per installation. The provisioning files now name the linked source instead, as they already do for the connection. provision.js writes every source without its links first, then again with each linked name resolved to that team's id, so sources may link to each other in either direction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Researchers had to switch to Langfuse for anything beyond counts: the analytics views left out conversation content, so HyperDX showed that a user rated a reply but not why, nor what was asked or answered. The views now carry it, so HyperDX covers what Langfuse records. - analytics.spans: every span from Langfuse's events_full (events_core truncates input and output to 200 characters), with full input and output, status, time to first token, and model settings, usage, cost, tool calls, tool definitions, tags and metadata as attributes. A new "Conversation traces" HyperDX source reads it in the trace layout, so a turn opens as a tree; the usage timeline links to it by TraceId. - analytics.chat_turns: each turn's message and reply, as `chat` rows on the timeline with the message as Body, so search finds it. - analytics.feedback: the reason (metadata['tag']), the comment with any text the user typed, and the score's metadata. - The timeline gains Reason, Comment, Input, Output, StatusMessage and TimeToFirstTokenMs; the LLM usage dashboard gains feedback by reason, feedback comments, conversations, tool calls, agent steps, errors and time to first token. This reverses the views' earlier content-free design: anyone who can sign in to HyperDX can now read what users wrote, the replies, tool inputs and outputs, and feedback comments. HyperDX still reads only the views, not Langfuse's tables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Added langfuse, clickhouse, otel, and hyperdxPlease ignore: this was meant to stay on my local fork and is irrelevant for the main project