Skip to content

Token replacement is slow because it rereads the template for every r… - #9352

Merged
bamaer merged 1 commit into
apache:mainfrom
hansva:9350
Nov 6, 2026
Merged

bamaer merged 1 commit into
apache:mainfrom
hansva:9350

Conversation

@hansva

@hansva hansva commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

…ow, fixes #9350

Please add a meaningful description for your change here


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Run mvn clean install apache-rat:check to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
  • If you have a group of commits related to the same change, please squash your commits into one and force push your branch using git rebase -i.
  • Mention the appropriate issue in your description (for example: addresses #123), if applicable.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

@bamaer

bamaer commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Thanks @hansva, LGTM.

What I checked:

  • Built the plugin and ran its unit tests: 21 pass.
  • Compared TokenTemplate with the old TokenReplacingReader on 300,000 random templates: nine marker pairs, including overlapping ones (%%/%%, aba/a), plus unclosed and unknown tokens, null values and values that contain markers. Same output in every case.
  • Timing, 3,200-character template with 3 tokens, 20,000 rows: 4.8 s before, 40 ms after.

Not run: integration test 0094.

@bamaer
bamaer merged commit 28fe41a into apache:main Nov 6, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Token replacement is slow because it rereads the template for every row

2 participants