Remove/update benchmarks by kunalmohan · Pull Request #288 · fjall-rs/lsm-tree

kunalmohan · 2026-04-19T17:18:09Z

The benchmarks were outdated and did not compile. This PR aims to update or remove the benchmarks.

Summary by CodeRabbit

Chores
- Removed several benchmark targets and retired multiple internal performance benchmarks to reduce maintenance.
- Cleaned up and simplified benchmark registrations and groups.
- Updated the remaining benchmark suites to use current initialization and API patterns for consistency.

coderabbitai · 2026-04-19T17:18:25Z

Warning

Rate limit exceeded

@kunalmohan has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 46 minutes and 41 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 46 minutes and 41 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9cf74527-3ce1-4a41-bf8a-66ceaab7d330

📥 Commits

Reviewing files that changed from the base of the PR and between 3f4c496 and 2410660.

📒 Files selected for processing (1)

benches/tree.rs

📝 Walkthrough

Walkthrough

Removed three explicit benchmark targets from Cargo.toml, deleted full benchmark files (tli, partition_point, block), removed blocked-bloom benchmarks, and updated several benchmarks to new APIs (Memtable construction, sequence-number usage, block-size policy, iterator typing).

Changes

Cohort / File(s)	Summary
Manifest `Cargo.toml`	Removed three `[[bench]]` declarations: `tli`, `block`, and `partition_point`. Remaining bench entries unchanged.
Removed benchmark files `benches/tli.rs`, `benches/partition_point.rs`, `benches/block.rs`	Deleted entire benchmark implementations and their `criterion_group!` / `criterion_main!` registrations.
Bloom benchmarks `benches/bloom.rs`	Removed blocked-bloom benchmark functions (`blocked_filter_construction`, `blocked_filter_contains`); left standard bloom benchmarks.
Memtable & merger updates `benches/memtable.rs`, `benches/merge.rs`	Replaced `Memtable::default()` with `Memtable::new(0_u64)`; adjusted `memtable.get` args (`None` → `u64::MAX`, `Some(1)` → `1`); added explicit iterator item type `Ok::<_, lsm_tree::Error>` in merger iterator construction.
Level manifest updates `benches/level_manifest.rs`	Switched `Config` construction to include `SequenceNumberCounter::default()` and `data_block_size_policy(BlockSizePolicy::all(1_024))`; replaced `tree.levels` counting with `tree.table_count()`; removed `find_segment` benchmark.
Tree benchmarks & SeqNo changes `benches/tree.rs`	Rewrote `Config::new(...)` calls to include sequence counters and `.use_cache(...)`; replaced `None` sequence args with `SeqNo::MAX` for `len`, `get`, `iter`, `prefix`; changed iterator/error handling to use `guard.into_inner().ok()?` and adjusted blob tree construction to use `.with_kv_separation(...)`.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Remove/update benchmarks' accurately reflects the main changes: several benchmark files were removed entirely while others were updated to fix compilation issues.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2026-04-19T17:20:28Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@benches/merge.rs`:
- Around line 9-11: The benchmark creates multiple Memtable instances with the
same ID (Memtable::new(0_u64)), violating the MemtableId uniqueness invariant;
change both maps that build memtables (the one assigning to variable memtables
and the second similar loop around lines 44-46) to supply distinct IDs (e.g.,
Memtable::new(i as u64) or a running counter) so each Memtable gets a unique
MemtableId instead of 0.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6174fb24-ebc3-42d0-b6e0-4e6357f59838

📥 Commits

Reviewing files that changed from the base of the PR and between 557cd0d and 071c2c2.

📒 Files selected for processing (7)

Cargo.toml
benches/bloom.rs
benches/level_manifest.rs
benches/memtable.rs
benches/merge.rs
benches/partition_point.rs
benches/tli.rs

💤 Files with no reviewable changes (4)

benches/bloom.rs
Cargo.toml
benches/tli.rs
benches/partition_point.rs

coderabbitai

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@benches/tree.rs`:
- Around line 16-23: The TempDir guard is being moved into Config::new which
drops the guard before .open() runs; update each Config::new call (e.g., the one
shown and the other setup blocks) to pass the Path reference instead of the
TempDir by using path.path() or folder.path() so Config::new receives a path
slice. Search for all Config::new(...) occurrences in benches/tree.rs (lines
around the shown blocks and the other listed blocks) and replace the first
argument from the TempDir variable (path or folder) to path.path() /
folder.path(); leave the rest of the call (SequenceNumberCounter::default(),
.use_cache(...), .open().unwrap()) unchanged.
- Around line 404-420: The benchmark inserts a large value into the tree but
doesn't flush, so the timed "blob tree get" measures memtable lookup rather than
blob storage; before starting the c.bench_function closure call the tree's flush
method (e.g., invoke tree.flush and handle/unpack its Result) to persist the
blob to storage, ensure the flush completed successfully, then run the timed
b.iter to measure blob read performance; reference:
Config::new/...with_kv_separation, tree.insert, and the "blob tree get" bench.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5cc8da96-9e93-40dd-bc2a-7bd8bda42042

📥 Commits

Reviewing files that changed from the base of the PR and between 071c2c2 and 3f4c496.

📒 Files selected for processing (3)

Cargo.toml
benches/block.rs
benches/tree.rs

💤 Files with no reviewable changes (2)

benches/block.rs
Cargo.toml

marvin-j97 · 2026-05-03T13:30:11Z

-                assert_eq!(levels.iter().count(), segment_count as usize);
+                assert_eq!(tree.table_count(), segment_count as usize);


levels.iter().count() was a very deliberate choice to force iterating the segments. count() is used to consume the entire iterator. table_count() obviously is an O(1) operation and masks the purpose of the benchmark.

marvin-j97 · 2026-05-03T13:32:27Z

I think though this file can be removed for now

Fix/update benchmarks

071c2c2

coderabbitai Bot reviewed Apr 19, 2026

View reviewed changes

Comment thread benches/merge.rs

fix benches with lz4 feature

3f4c496

coderabbitai Bot reviewed Apr 21, 2026

View reviewed changes

Comment thread benches/tree.rs

Comment thread benches/tree.rs

address review comments

2410660

kunalmohan mentioned this pull request Apr 26, 2026

Feat: Add batch write implementation #290

Open

marvin-j97 reviewed May 3, 2026

View reviewed changes

Comment thread benches/level_manifest.rs

Copy link
Copy Markdown

Contributor

marvin-j97 May 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think though this file can be removed for now

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Remove/update benchmarks#288

Remove/update benchmarks#288
kunalmohan wants to merge 3 commits intofjall-rs:mainfrom
kunalmohan:fix/benchmarks

kunalmohan commented Apr 19, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Apr 19, 2026 •

edited

Loading

Rate limit exceeded

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

codecov Bot commented Apr 19, 2026

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

marvin-j97 May 3, 2026

Uh oh!

marvin-j97 May 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		assert_eq!(levels.iter().count(), segment_count as usize);
		assert_eq!(tree.table_count(), segment_count as usize);

Uh oh!

Conversation

kunalmohan commented Apr 19, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rate limit exceeded

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

codecov Bot commented Apr 19, 2026

Codecov Report

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

marvin-j97 May 3, 2026

Choose a reason for hiding this comment

Uh oh!

marvin-j97 May 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kunalmohan commented Apr 19, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Apr 19, 2026 •

edited

Loading