Workspace aware sharing records - #6374
Conversation
Let a resource belong to multiple workspaces and be discoverable by workspace members through the existing DLS sharing mechanism. Adds a workspace: principal namespace: workspace IDs on a resource are projected into all_shared_principals, and a user's accessible workspaces are added to the DLS filter, so the existing terms intersection grants visibility via workspace membership with no new query shape. Spike scope (read/discovery path only): - SPI: ResourceProvider.workspacesField() (default null; additive) - Multi-value field extraction from the index op at index time - ResourceSharing.workspaces set: builder, XContent (omitted when empty), fromXContent, equals/hashCode/toString, version-guarded writeTo - getAllPrincipals() emits workspace:<id>; DLS adds the user's workspaces I/O-free from an in-memory User attribute (honors hot-path no-I/O rule) - Seed visibility from getAllPrincipals() (creator + workspaces) Not yet addressed (follow-ups, intentionally not stubbed): - Write path (hasPermission) cross-record resolution of workspace access levels from the workspace's own sharing record - WORKSPACES_INTRODUCED_VERSION is a compile-only placeholder - No registered NamedWriteable reader for ResourceSharing (pre-existing) - Lucene doc-values materialization needs an integration-test spike - User->workspaces attribute key and authc-time population are placeholders ResourceSharingTests: 21 tests, 0 failures. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
Grant a user access to a resource when they have the required access level on any workspace the resource belongs to, not just when the resource is shared with them directly. Generalizes the existing single-parent access recursion in hasPermission into a fan-out over the resource's containers: its hierarchical parent (if any) plus each of its workspaces. Each workspace is resolved through hasPermission against the workspace's own sharing record, so workspace collaborators and their access levels map through the workspace type's action groups (per issue opensearch-project#6119). Access is granted if any container grants it; evaluation short-circuits on the first grant. Spike notes / follow-ups: - Workspace resource type name is a placeholder ("workspace"); the real type comes from the workspace provider registered via the SPI. If no provider is registered, the workspace branch denies cleanly. - No cycle/depth guard yet; safe for the intended model (workspace records do not themselves carry workspaces) but should be added. ResourceAccessHandlerTests: 15 tests, 0 failures. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
Prevent unbounded recursion when a resource inherits access from its containers (parent and/or workspaces) and the container graph is malformed (e.g. a workspace that transitively contains itself). Threads a visited set of type:id keys through the permission walk; re-encountering an already-visited resource short-circuits to false, which is safe under the fan-out's OR semantics. The public hasPermission signature is unchanged; a private overload carries the set. ResourceAccessHandlerTests: 16 tests, 0 failures (adds a self- referential-workspace cycle case). Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
Resource sharing was introduced in 3.3 and is not yet GA, and the workspaces field ships within that same not-yet-released feature, so no older node speaks a wire format that omits it. The version gate (and its placeholder constant) added nothing but a misleading TODO; serialize the field unconditionally. The pre-existing NamedWriteable reader gap for ResourceSharing is unchanged and still noted as a follow-up. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
Avoid an N+1 sequential-GET pattern when a resource inherits access from the workspaces it belongs to. Previously each container was resolved by a separate recursive hasPermission call, i.e. one GET per workspace, serially, on the privilege hot path. Fetch all of a resource's workspace sharing records in a single mget (they live in one index with known ids) and evaluate them in memory via a new pure recordGrantsAction helper. The single hierarchical parent is still resolved recursively so grandparent chains keep working, and the visited-set cycle guard now also pre-filters workspace ids before the batch. Workspace records are evaluated as leaves (their own share_with), matching the flat workspace model. ResourceAccessHandlerTests: 16 tests, 0 failures. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
PR Code Analyzer ❗AI-powered 'Code-Diff-Analyzer' found issues on commit 533c69a. ⛔ Hard block: Issues at High severity or above will block this PR from merging.
The table above displays the top 10 most important findings. Pull Requests Author(s): Please update your Pull Request according to the report above. Repository Maintainer(s): You can Thanks. |
PR Reviewer Guide 🔍(Review updated until commit 649432f)Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Latest suggestions up to 649432f Explore these optional code suggestions:
Previous suggestionsSuggestions up to commit 8f62005
Suggestions up to commit 81f704c
Suggestions up to commit 7c5cf51
Suggestions up to commit eca8bdf
Suggestions up to commit 1577069
|
Make the resource-sharing migrate endpoint workspace-aware so workspaces that predate resource sharing carry their membership into the sharing records created during migration. When a provider declares workspacesField(), read the (multi-valued) set of workspace IDs off each source-doc search hit and set it on the built ResourceSharing record, so getAllPrincipals() emits workspace:<id> and DLS/write-path inheritance work for backfilled records exactly as for records indexed while the feature is on. Providers that do not declare the field are unaffected. Extraction is factored into a package-private static extractWorkspaces helper (array or scalar, blank ids ignored, dot-notation paths), the migrate-path counterpart of ResourcePluginInfo.extractMultiValuedField- FromIndexOp. Does not address updating already-migrated (skippedExisting) records or materializing workspace collaborator records from frontend ACLs; both are tracked as follow-ups. MigrateResourceSharingInfoApiActionTests: 13 tests, 0 failures. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
ac9b426 to
7412d76
Compare
|
Persistent review updated to latest commit 7412d76 |
Two findings from the PR code analyzer: 1. (Medium, security) DLS resolved workspace membership from a user-influenceable custom attribute, which feeds authorization and could let a user claim arbitrary workspace membership and read those workspaces' resources. Since no trusted server-set source of membership is wired yet, disable the resolver (returns empty) with an explicit server-set-only contract, removing the escalation vector until the trusted mechanism exists. 2. (Robustness) The container cycle guard used a global visited set and denied re-entry, which could falsely deny a node reachable from more than one branch in a DAG. Scope the guard to the current ancestor (parent) chain and remove each key when its node resolves; workspaces are leaf-evaluated and no longer touch the set at all, so sibling branches can never falsely deny each other. Resources test package: 95 tests, 0 failures. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
…ring-records Signed-off-by: Darshit Chanpura <dchanp@amazon.com> # Conflicts: # src/main/java/org/opensearch/security/resources/ResourceAccessHandler.java # src/main/java/org/opensearch/security/resources/ResourceIndexListener.java
|
Persistent review updated to latest commit 0a7486f |
ShareResponse deserializes ResourceSharing via readNamedWriteable(ResourceSharing.class), but the class had no StreamInput constructor and was not registered in the plugin's NamedWriteable registry, so any transport round-trip of a ShareResponse would fail. Add a StreamInput constructor symmetric with writeTo (including the workspaces field), expose a NAME constant, and register the reader in OpenSearchSecurityPlugin#getNamedWriteables. Adds stream round-trip tests (with and without workspaces); these assert fields explicitly since CreatedBy/ShareWith use identity equality. ResourceSharingTests: 23 tests, 0 failures. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit 5333eba |
Migration indexes with OpType.CREATE, so resources that were migrated for ownership before workspace-awareness existed are skipped and left without workspace membership -- invisible via workspace-based DLS and with nothing for the write-path fan-out to inherit. When migration skips an existing record but the source doc declares workspaces, backfill instead of skipping: merge the workspace IDs onto the existing sharing record and refresh all_shared_principals. The new ResourceSharingIndexHandler#backfillWorkspacesOnExisting is idempotent (a no-op when the workspaces are already present) and leaves created_by and share_with untouched. Migration now reports a backfilledExisting count distinct from skippedExisting. Coverage note: like the rest of the async index-handler flow, this is exercised via integration tests (tracked follow-up), not unit tests; the pure read side (extractWorkspaces) is already unit-tested. Resources test package: 101 tests, 0 failures (no regressions). Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit 19af1df |
Cover the migrate workspace-backfill path end-to-end against a real cluster: create a resource whose sharing record already exists, add workspace membership to its source doc, then migrate. Asserts the record is backfilled (not skipped), the workspaces field is persisted, and all_shared_principals gains the workspace:<id> entries; a second migrate is a no-op (idempotent). Declares workspacesField() on the sample resource provider so the migrate/index paths can read workspace membership. Also updates the migrate summary-string assertions in existing tests for the new backfilledExisting count. MigrateApiTests: 17 tests, 0 failures (live cluster). Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit 3115d4c |
Cover steady-state workspace-awareness end-to-end on a real cluster: creating a sample resource with a workspaces field must trigger ResourceIndexListener to extract the (multi-valued) IDs from the parsed doc and project workspace:<id> into all_shared_principals plus the workspaces field on the sharing record -- with no migrate call. Answers the Lucene getFields() materialization question empirically for a default (dynamic) mapping. Threads workspaces through the sample resource so the test can create a resource carrying them: - SampleResource: optional Set<String> workspaces field with builder- compatible getter/setter, additive XContent (emitted only when non- empty so pre-existing docs stay byte-identical), parser, and StreamInput/writeTo symmetry. - CreateResourceRestAction: read workspaces off the request body's Map<String,Object> in both create and update paths. MigrateApiTests: 18 tests, 0 failures. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
Replace the placeholder in ResourceSharingDlsUtils (which returned empty "until a trusted server-set source is wired") with a real SPI extension point. Plugins that own an authoritative workspace-membership store implement resolveWorkspacesForUser on their ResourceSharingExtension; ResourcePluginInfo aggregates the contributions across all registered extensions, and the DLS builder projects them as workspace:<id> principals. The SPI contract is explicit in the javadoc: results MUST come from a trusted server-set source (not user-assertable via JWT/proxy claims) and the call MUST be I/O-free (privilege hot path). Default returns empty, so plugins that don't opt in are unaffected -- and unimplemented remains safe by default (no privilege-escalation vector). Sample plugin implements the resolver by mapping security roles to deterministic workspace IDs, giving the read-path SPI end-to-end coverage. ResourcePluginInfoTests: 11 tests, 0 failures. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
Raise patch coverage on the workspace-aware changes with unit tests for code that was previously exercised only by integration tests (which CI does not merge into the Codecov patch report): - New ResourceSharingIndexHandlerTests mocks the Client to cover fetchSharingInfoForIds (mget parse + skip-missing) and backfillWorkspacesOnExisting (empty/missing/already-present no-ops and the merge+refresh update path). - ResourcePluginInfoTests: cover extractMultiValuedFieldFromIndexOp (multi-value collect + empty-when-absent). No production changes. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit 31fde47 |
Open items / scope notes for reviewersIn-repo, tracked as follow-ups (not blocking this PR's mechanics):
Cross-repo dependencies (cannot land here):
Decisions for the team (not code):
Happy to split any of the above into tracking issues if preferred. |
Decision needed: where does the OpenSearch-side workspace backend live?The
Not the security plugin's responsibility either way. Owner: workspaces team. This gates the read-path resolver going live (it is inert until implemented) and ties into the #6348 graduation gate. |
Follow-up: dynamic workspace membership updates (associate/dissociate)OSD shares an object to another workspace by mutating its
Not addressed in this PR (create + migrate-backfill are). Fix: on update, re-extract the object's |
| } | ||
|
|
||
| @Test | ||
| public void testMigrateBackfillsWorkspacesOntoExistingRecord() { |
There was a problem hiding this comment.
If I'm reading this correctly, the migrate API can now be called multiple times? One from a version before this PR and one afterwards to get the workspace info?
| * @param backendRoles the user's backend roles; never {@code null}, may be empty | ||
| * @return the trusted workspace IDs the user belongs to, or an empty set if none / not implemented | ||
| */ | ||
| default Set<String> resolveWorkspacesForUser(String username, Set<String> securityRoles, Set<String> backendRoles) { |
There was a problem hiding this comment.
@DarshitChanpura With multi-tenancy, with a request destined for a tenant there is a securitytenant header. What is the equivalent with workspaces? If a request is for a particular workspace then there would only be a single workspace to filter by for security.
There was a problem hiding this comment.
Deliberately no securitytenant-style header for workspaces — access is membership-scoped, not request-scoped.
The models differ:
securitytenantworks as a request selector because a saved object lives in exactly one tenant, and the tenant is the security boundary — the header picks which tenant's index you operate in.- A workspace here is a container a resource belongs to, and a resource can be in multiple workspaces (1:N). The security boundary is the user's membership across all their accessible workspaces, resolved server-side via the trusted
resolveWorkspacesForUser— not something the request asserts.
So on the read path, DLS filters the resource's workspaces field against the user's whole accessible-workspace set (a terms over the set), not a single header-selected workspace.
A "current workspace" (the workspace the UI is focused on) is the closest analogue to securitytenant, but it's a UX/narrowing concern, not a security one: such a header could only narrow results to that workspace (intersected with membership) — it can never grant, since membership is the enforced floor. DLS already works with no header, so it's an optional fast-follow if the UI wants server-side scoping to the active workspace.
Worth flagging: because membership is 1:N (unlike 1:1 tenant), "a single workspace to filter by" isn't quite the model even when a request targets one workspace — security still authorizes against the user's full membership; a single-workspace header would be an extra filter on top.
There was a problem hiding this comment.
But for requests like GET Report Definition, how does that request ferry along which workspace the user is toggled to in dashboards to the backend?
Per review (cwperks): - Drop the ancestor cycle guard in ResourceAccessHandler. Workspaces are leaf-evaluated (never recursed), and parent inheritance is a single chain the pre-existing code already recursed without a guard, so the visited-set + path-scoping added complexity for a case that cannot occur. hasPermission is back to a single method. - ResourceProvider.workspacesField() now defaults to "workspaces" (a doc without the field belongs to no workspace), and its javadoc no longer references the removed all_shared_principals denormalization. - Trim code comments that referenced prior revisions; drop the now- redundant workspacesField override in the sample plugin. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Thanks @cwperks — addressed in e2f4225:
|
|
Persistent review updated to latest commit e2f4225 |
|
On request-scoping (your
No code change needed — the current DLS already filters the |
Tighten javadoc/inline comments on the workspace-awareness code: drop spike/placeholder framing and intra-PR history, correct stale mentions of workspace principal denormalization (read path filters the workspaces field), and shorten multi-line blocks. Comments only; no behavior change. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit d86d9c6 |
Reconcile ResourceSharing.workspaces to the resource doc's exact set on every primary write, including removals. Previously the listener only stamped workspaces on create; updates refreshed principals but left the sharing record unchanged, so after a dissociation the write path could still authorize via a stale workspace while the read path (which filters the live doc field) had already dropped visibility. - Listener reconciles the record on each update (add and remove). - Migration reconciles pre-existing records to the doc exactly, rather than union-merging (backfillWorkspacesOnExisting -> reconcileWorkspaces). - Resolve the per-index workspaces field deterministically instead of by map iteration order, warning on conflicting declarations. - Require the workspaces field to be keyword-mapped for the DLS terms filter; declare it on the sample resource index. - Tests: live deny/allow/deny for DLS visibility plus the record reconcile; unit coverage for exact reconcile and write-path grant loss after dissociation. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit 725dfb3 |
Exercise the write-path container fan-out end-to-end. The sample plugin now registers "workspace" as a resource type (sharing the sample index, distinguished by resource_type) with workspace_read_only/read_write/ full_access levels that map to child sampleresource actions, mirroring the existing resource-group container. Adds a live deny/allow/deny test: a user with no direct grant can GET a resource only while it belongs to a workspace shared with them, and loses access the moment it is dissociated -- covering checkContainers and confirming the sharing record's workspace set is reconciled (added and removed) on every write. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit 361ec6b |
Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit 1577069 |
Address PR review findings on workspace-aware resource sharing. - Reconcile the sharing record's workspaces monotonically: the source document's seq_no is stored as workspaces_seq_no and a reconcile is applied only when newer, guarded by if_seq_no/if_primary_term with retry. This stops a slow reconcile from overwriting a newer associate or dissociate, and retries a not-yet-created record instead of treating it as synced. - Document the trusted-write contract: workspace membership drives authorization, so an ordinary resource update must not change it. The sample update route now ignores caller-supplied workspaces, and an integration test proves a workspace_read_write user cannot add a workspace to escalate. - Reject conflicting workspaces-field declarations across providers sharing an index at registration, rather than silently choosing one. - Tests: negative-control associate-then-dissociate and create-then-clear races that converge to denied; unit coverage for the monotonic guard and the registration conflict. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit eca8bdf |
Address follow-up review on the workspace reconciliation guard. - Model workspaces_seq_no on ResourceSharing (builder, XContent, transport) so the monotonic guard survives whole-record rewrites. share()/revoke()/patch() re-index via toXContent, which previously dropped the field and reset the guard, letting an older still-retrying reconcile re-apply stale workspaces. Round-trip + guard-survives-share tests added. - Narrow the trusted-write contract wording: the escalation constraint is on ordinary updates of existing resources; create-time placement is owner-governed but a real backend must still validate the creator can add to each workspace. - Reconcile writes with WAIT_UNTIL instead of IMMEDIATE (write-path reads are realtime), and on give-up logs an error pointing to migrate for repair instead of a silent warn. - Strengthen the missing-record test to verify a retry is scheduled. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit 7c5cf51 |
share() and patchSharingInfo() fetched a whole sharing record, mutated it, and re-indexed the snapshot with an unversioned INDEX. A workspace reconcile committing between the fetch and the re-index was silently reverted (its newer workspaces and guard overwritten by the stale snapshot). Make both paths optimistic-concurrency safe: capture the record's _seq_no/_primary_term at fetch, write with if_seq_no/if_primary_term, and on version conflict re-fetch and re-apply the mutation to the latest record. A concurrent reconcile's workspaces/workspaces_seq_no now survive. Adds a deterministic unit test that interleaves a stale share fetch, a conflicting write (reconcile won), and the retry, asserting the re-index carries the newer empty workspaces and guard rather than the stale set. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit 81f704c |
…S test Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit 8f62005 |
Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
|
Persistent review updated to latest commit 649432f |
| // ResourceSharingExtension.resolveWorkspacesForUser, whose SPI contract requires a trusted, server-set, | ||
| // I/O-free source (see that interface's javadoc). Filtering the live field means associate/dissociate are | ||
| // reflected automatically. If no extension implements the resolver, the set is empty and the clause is omitted. | ||
| Set<String> userWorkspaces = resourcePluginInfo == null ? Set.of() : resourcePluginInfo.resolveWorkspacesForUser(user); |
There was a problem hiding this comment.
I think we need to figure this out. With multi-tenancy, we set user.getRequestedTenant to know which tenant a request is destined for.
There was a problem hiding this comment.
I think security-dashboards-plugin carries the tenant info via https://github.com/opensearch-project/security-dashboards-plugin/blob/d4db9d33dde68e8c62565bfbd7bea66e980399eb/server/auth/types/authentication_type.ts#L175-L193
Description
Category: Enhancement
Makes the resource-sharing framework workspace-aware, so a resource's visibility and access can be driven by the workspaces it belongs to. This is the compatibility prerequisite for graduating resource sharing (RP) out of experimental — workspaces converge onto the RP ownership model rather than the reverse.
What changed
ResourceSharinggains an optional, set-valuedworkspacesfield (a resource may belong to multiple workspaces). Threaded through the builder, XContent (omitted when empty → byte-identical for existing records), and transport serialization. Also adds the previously-missingStreamInputreader +NamedWriteableregistration soShareResponseround-trips.workspace:<id>principal namespace. Workspace IDs on a resource are denormalized intoall_shared_principals; a user's accessible workspaces are added to the DLStermsfilter — so visibility via workspace membership works through the existing intersection with no new query shape.hasPermissioninto a container fan-out (hierarchical parent + workspaces). Workspaces are batched in a singlemget(avoids N+1 GETs on the privilege hot path) and evaluated in-memory; an ancestor-scoped cycle guard prevents unbounded recursion.ResourceIndexListenerreads the multi-valued workspaces field off the indexed doc; migrate API is workspace-aware and adds an idempotent backfill path for already-migrated records (transition support for workspaces that predate RP).ResourceSharingExtension.resolveWorkspacesForUser(...)extension point for trusted, server-set workspace membership. Default returns empty (safe by default); the contract explicitly requires the source to be non-user-assertable and I/O-free.Old vs. new behavior: non-workspace resources are unaffected — sharing records are byte-identical (the
workspacesfield is omitted when empty), and the workspace resolver is disabled unless a plugin opts in. The DLS filter's shape changes from a singletermsto abool.should(min_should_match=1); with an empty resolver the workspace clause is dropped, so it stays semantically equivalent (same documents match) though the query JSON differs. New behavior only activates for workspace-associated resources when a trusted resolver is registered.Issues Resolved
Relates to #6348 (graduate resource sharing out of experimental) and #6119 (backend workspace collaborator management). No issue fully closed by this PR — see the open items comment below.
Not a backport.
No new static front-end permissions introduced.
Testing
ResourceSharing,ResourceAccessHandler,ResourcePluginInfo,ResourceSharingIndexHandler(incl. the mget + backfill paths), and the migrate extraction.sample-resource-plugin: migrate workspace-backfill (idempotent) and live-index ingestion (which empirically confirms the workspace field is read at index time on the default mapping).resourcesunit package green; Codecov patch passing.Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.