Fix: Orca Security Alerts Connector misses alerts that become eligible after creation - #1185
Open
bahdanb-orca wants to merge 8 commits into
Open
Fix: Orca Security Alerts Connector misses alerts that become eligible after creation#1185bahdanb-orca wants to merge 8 commits into
bahdanb-orca wants to merge 8 commits into
Conversation
The JSON result example validation expects resources/<ActionName>_JsonResult_example.json, matching the action's script file name. These files used snake_case names, so the validation failed for every action with a JSON result. Rename the six example files and update the result_example_path in each action definition accordingly. No content changes.
Both window bounds now appear in the run log, so the fetch window can be diagnosed from the logs alone instead of being inferred from which alerts came back.
bahdanb-orca
marked this pull request as ready for review
August 26, 2026 10:11
…mary Drops implementation detail from the release note per review feedback, leaving a high-level description of the fix.
Author
|
Hello @KrishnaSharma06 I've addressed your comment regarding notes, could you please check it out? |
KrishnaSharma06
approved these changes
Sep 7, 2026
KrishnaSharma06
requested changes
Sep 7, 2026
Comment on lines
+178
to
+179
| - description: Orca Security - Alerts Connector - Fixed an issue where alerts becoming | ||
| eligible after creation were intermittently missed. |
Contributor
There was a problem hiding this comment.
Description has to be aligned in single line, please fix it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
What problem does this PR solve?
The Orca Security Alerts Connector intermittently fails to ingest alerts. Two independent causes were found:
Alerts that become eligible after creation are lost permanently. The connector advanced its watermark over
CreatedAt. An alert whose Orca Score is populated after creation (for example by a score override) only starts matching the connector'sLowest Orca Score To Fetchfilter later — by which time the watermark has already moved past itsCreatedAt, so it never enters the fetch window again.The
date_rangefilter is day-granular, so the watermark could never advance within a day. The range start is rounded up to the next UTC midnight (verified empirically: a start of00:00:00.000matches that day,00:00:01matches nothing until the next day). Combined with a server-sidelimitand no pagination, this made the connector behave like a once-a-day batch that silently dropped the tail of any day with more alerts thanMax Alerts To Fetch.How does this PR solve the problem?
connectors/AlertsConnector.py— the fetch position is now alast_synccursor (the alert row's database write time), so an alert re-enters the window whenever its row becomes visible again. Late score population is handled by construction rather than by a heuristic buffer. ACreatedAt >= now - 3hbound preserves "new alerts only" semantics when resuming, so rewrites of older alerts (rescans, status changes) never re-enter the window. The first run is not restricted by that bound and honoursMax Hours Backwards.core/query_builder.py— both time filters now use the datetime-preciserangeoperator with ISO 8601 values instead of the day-granulardate_range, and results are ordered bylast_syncwhen that cursor is in use.connectors/AlertsConnector.py— cursor pagination: full pages advance the cursor to the page's maximumlast_sync; a full page confined to a singlelast_syncsecond pages deeper withstart_at_indexrather than skipping the remainder of that second; and the watermark advances over pages that turned out to be all duplicates, so progress no longer depends on finding new alerts.core/datamodels.py,core/OrcaSecurityParser.py— parselast_sync, falling back to the creation time when the field is absent.core/constants.py— the duplicate-suppression ID cache is raised to 10,000, and the connector logs when it overflows.tests/— 18 new tests covering the connector loop and the query/response contract.pyproject.toml,uv.lock,release_notes.yaml— integration version bumped 15.0 to 16.0 with a matching release-notes entry.Any other relevant information (e.g., design choices, tradeoffs, known issues):
limitserver-side before client-side ID de-duplication, so if a buffer window ever contains more already-seen alerts than the page size, every run returns the same first page, all of it is discarded as duplicates, and the watermark never advances — a permanent stall. Advancing the watermark across duplicate-only pages is what avoids this.order_by CreatedAthas no deterministic tiebreaker and the range end is re-evaluated per request, so offset pages can silently skip rows under concurrent inserts — the very failure being fixed here. Offsets are therefore used only to page within a singlelast_syncsecond, back-to-back inside one run, never carried across runs.last_syncrewrite, unless itsCreatedAthas meanwhile aged out of the lookback window. This is documented in a code comment.last_syncis not part of the schema published by/schema. The behaviour relied on here — that the field is filterable, orderable, returned in responses, wrapped in{"value": ...}, and timezone-aware at second resolution — was verified against a live tenant, and the test fixtures pin that shape so a change would fail the test suite rather than fail silently at runtime.JSON Result Example Validationexpectsresources/<ActionName>_JsonResult_example.json, matching the action's script file name, while these files used snake_case names. The mismatch is pre-existing, but it makesmp validateexit non-zero for this integration, so the validation job cannot go green without it. It is kept as a separate commit — happy to split it into its own PR if you prefer.mp validate integration orca_security— 23/23mp test --integration orca_security— 18/18mp build --integration orca_security— succeedsstart_at_indexpaging inside a tiedlast_syncsecond, recovery of an alert whose score was raised after the watermark had passed itsCreatedAt, rejection of an updated alert created outside the lookback window, and exactly-once delivery across consecutive runs.Checklist:
General Checks:
Open-Source Specific Checks:
Screenshots (If Applicable)
Not applicable — no UI or visual changes.
Further Comments / Questions
CreatedAtlookback is the window in which an alert can still be recovered after becoming eligible. Let us know if you would prefer a different default, or the value exposed as a connector parameter.