Skip to content

Support continuous scan mode on RDS source - #7181

Merged
graytaylor0 merged 1 commit into
opensearch-project:mainfrom
graytaylor0:RdsContinuousScanMode
Oct 5, 2026
Merged

graytaylor0 merged 1 commit into
opensearch-project:mainfrom
graytaylor0:RdsContinuousScanMode

Conversation

@graytaylor0

@graytaylor0 graytaylor0 commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Description

Following from this change for DocumentDB source, but enabled for RDS as it uses the same architecture: #7172

Usage

rds

source:
  rds:
    scan_mode: "continuous"
     ....

s3 source

source:
  s3:
    codec:
      event_json:
    compression: "none"
    aws:
      region: "us-east-1"
      sts_role_arn: "arn:aws:iam::123456789012:role/MyRole"
    acknowledgments: true
    delete_s3_objects_on_read: true
    scan:
      folder_partitions:
        depth: 3
        max_objects_per_ownership: 50
        idle_scan_mode: "continuous"    # NEW — "idle_optimized" (default) or "continuous"

Issues Resolved

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Taylor Gray <tylgry@amazon.com>

@dlvenable dlvenable left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@graytaylor0
graytaylor0 merged commit c496f1b into opensearch-project:main Oct 5, 2026
36 of 75 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants