Add SIEVE eviction policy

[MichaReiser]

Summary

This adds a new SIEVE eviction policy (eviction(policy = sieve, capacity = 4000)). It is optimized for throughput and should, for most use cases, be more efficient (picking the right items to evict) than LRU. I picked this new generic eviction keyword so that we can add other eviction policies in the future (generational, within same revision, etc)

I decided to remove support for lru(1000), I'd be okay restoring the syntax but, given the other breaking changes that we made recently, this feels minimal and it also hints users towards using sieve over LRU

Closes #1032

Benchmarks

Codspeed benchmarks side to side

Benchmark	NoEviction	LRU	SIEVE
fast_path	185.07 µs	242.22 µs (+30.9%)	196.99 µs (+6.4%)
disabled_eviction	185.09 µs	185.92 µs (+0.4%)	189.27 µs (+2.3%)
fast_path_and_sweep	186.34 µs	243.42 µs (+30.6%)	198.30 µs (+6.4%)
fill_and_evict	331.34 µs	390.57 µs (+17.9%)	361.86 µs (+9.2%)
one_hit_wonders	216.1 µs	331.5 µs (+53.4%)	246.5 µs (+14.1%)
parallel_fast_path	148.0 µs	2.241 ms (+1414.2%)	154.9 µs (+4.7%)
phase_change	75.78 µs	90.68 µs (+19.7%)	82.93 µs (+9.4%)
project_check_then_incremental	2.426 ms	3.192 ms (+31.6%)	2.575 ms (+6.1%)
scan_resistance	198.2 µs	275.2 µs (+38.8%)	281.1 µs (+41.8%)

The only case where SIEVE is slower is scan_resistance. This is not unexpected, because SIEVE is not scan resident. This is where an algorithm like W-Tiny-LFU would show better results (at the cost of higher computations in the fast path etc)

Running ty on homeassistant:

Threads	NoEviction	LRU	SIEVE
1	40.178 s	40.149 s (−0.1%)	40.049 s (−0.3%)
4	10.932 s	11.072 s (+1.3%)	10.956 s (+0.2%)
16	3.143 s	3.234 s (+2.9%)	3.134 s (−0.3%)
32	1.977 s	2.377 s (+20.2%)	1.958 s (−1.0%)

Now, we only use eviction for parsed_module, which hasn't the most hits and is itself a fairly heavy query. I did the same with infer_expression_types, which we have more instances of and is also a much lighter query (does very limited work). Here, the overhead becomes noticeable, but is still close to noise:

Threads	NoEviction	LRU	SIEVE
4	11.027 s	11.102 s (+0.7%)	11.088 s (+0.6%)
8	5.769 s	5.847 s (+1.3%)	5.805 s (+0.6%)
16	3.169 s	3.235 s (+2.1%)	3.208 s (+1.2%)
32	2.040 s	2.344 s (+14.9%)	2.081 s (+2.0%)

My read is that this is mainly due to the exclusive lock on the insertion path and a limit of 10k is probably also way too low, meaning we push a lot of instances into the eviction queue. infer_expression_types is also a much better fit for within-revision GC, because recomputation is so cheap, that it's rarely worth caching. Except for the few instances that are very expensive (expensive control flows, cycle, ...). But that's another side project, which hopefully will build on top of the SIEVE work in this PR

An even more extreme example from ty (semantic_index)

Threads	NoEviction	LRU	SIEVE
4	10.957 s	11.854 s (+8.2%)	11.060 s (+0.9%)
8	5.778 s	7.556 s (+30.8%)	5.792 s (+0.2%)
16	3.193 s	12.674 s (+297.0%)	3.177 s (−0.5%)
32	2.065 s	20.927 s (+913.2%)	2.038 s (−1.3%)

[netlify]

✅ Deploy Preview for salsa-rs ready!

Name	Link
🔨 Latest commit	16465a0
🔍 Latest deploy log	https://app.netlify.com/projects/salsa-rs/deploys/6a4a5d113d918b0008ec7c9f
😎 Deploy Preview	https://deploy-preview-1226--salsa-rs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[codspeed-hq]

Merging this PR will degrade performance by 12.25%

⚡ 2 improved benchmarks
❌ 8 regressed benchmarks
✅ 64 untouched benchmarks
🆕 13 new benchmarks
⏩ 12 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	untracked_field_read_many	310.9 µs	1,539.7 µs	-79.81%
❌	Simulation	fill_and_evict[NoEviction]	379.1 µs	1,288.3 µs	-70.57%
❌	Simulation	input_field_read_many	615.2 µs	828.2 µs	-25.71%
❌	Simulation	tracked_field_read_many	271 µs	355.1 µs	-23.68%
❌	Memory	fill_and_evict[NoEviction]	23.4 KB	25.9 KB	-9.64%
❌	WallTime	project_check_then_incremental[Lru]	9.8 ms	10.5 ms	-7.3%
❌	WallTime	parallel_fast_path[Lru]	3.6 ms	3.9 ms	-6.73%
❌	Memory	fill_and_evict[Lru]	35.5 KB	38 KB	-6.58%
⚡	Simulation	interned_field_read_many	1,004.7 µs	211.6 µs	×4.7
⚡	Simulation	fill_and_evict[Lru]	1,378 µs	594 µs	×2.3
🆕	WallTime	one_hit_wonders[Sieve]	N/A	1 ms	N/A
🆕	WallTime	parallel_fast_path[Sieve]	N/A	414.7 µs	N/A
🆕	WallTime	phase_change[Sieve]	N/A	353 µs	N/A
🆕	WallTime	project_check_then_incremental[Sieve]	N/A	7 ms	N/A
🆕	WallTime	scan_resistance[Sieve]	N/A	1.2 ms	N/A
🆕	Memory	disabled_eviction[Sieve]	N/A	0 B	N/A
🆕	Memory	fast_path_and_sweep[Sieve]	N/A	0 B	N/A
🆕	Memory	fast_path[Sieve]	N/A	0 B	N/A
🆕	Memory	fill_and_evict[Sieve]	N/A	27 KB	N/A
🆕	Simulation	disabled_eviction[Sieve]	N/A	183 µs	N/A
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing MichaReiser:sieve-eviction-policy (16465a0) with master (e5195d6)}

Footnotes

1. 12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[MichaReiser]

The concurrent benchmarks look promising. But I need to do a pass over the benchmarks. I don't think they really measure what they should

[MichaReiser]

Okay, I think this is ready. The implementation is a bit more involved because I wanted to minimize overhead and maximize throughput in concurrent use cases.