Flexible memo dropping behavior

[Veykril]

This allows users to opt-into receiving a channel receiver on storage creation that can be used to decide when and where to drop destructors of stale memo results.

Notably this does not yet handle LRU evictions, so those are still synchronous. Handling that is more complicated or will have to incur allocations so I am not yet sure how to best approach this. We could maybe consider a double buffer (triple buffer) approach for the deletion queues per ingredient instead and if those are exhausted fall back to blocking in case of too frequent revision bumps. That is something I will consider in a separate follow up PR though, for now this is ready for a review.

Note that the perf changes are noise due to allocation order differences.

[netlify]

✅ Deploy Preview for salsa-rs canceled.

Name	Link
🔨 Latest commit	7ab9dba
🔍 Latest deploy log	https://app.netlify.com/projects/salsa-rs/deploys/68fe6b9ae26eea0008bd334f

[codspeed-hq]

Merging this PR will degrade performance by 11.01%

❌ 1 regressed benchmark
✅ 3 untouched benchmarks
🗄️ 9 archived benchmarks run¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	many_tracked_structs	8.8 µs	9.9 µs	-11.01%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing Veykril:veykril/push-proytsnlytqr (7ab9dba) with master (e8ddb4d)}

Footnotes

1. 9 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩

[davidbarsky]

Three things:

1. I didn't know it was legal to write unsafe fn(NonNull<DummyMemo>). I'm guessing that if you pass it a safe closure, this will fail to compile?
2. Can you make these named fields instead of tuple structs?
3. I really think this should be tested with Shuttle. if you're okay with me pushing to your branch, i can do that?

[Veykril]

I didn't know it was legal to write unsafe fn(NonNull). I'm guessing that if you pass it a safe closure, this will fail to compile?

No, safe functions/closures can be coerced to unsafe ones just fine. After all calling a safe function in an unsafe context isn't problematic. Only the other direction won't work for obvious reasons.

The other two questions won't be relevant anymore once im finished writing out the new approach

[Veykril]

Okay, new approach. The user is now in charge of things. We now have two dropping modes, synchronous (what we currently have) and channel based, where the user will receive a receiver on storage construction that they can poll to drop things as they like.

[Veykril]

So, delaying the LRU clears is actually a bit more difficult and as it turns out, my previous approach was even unsound. We need to remove the values immediately, as otherwise we might be racing with computations of the next revision. That is a bit unfortunate though, as the values are not necessarily allocated and so we can't just take the pointer as an opaque droppable thing. That leaves two options, allocate LRU'd values that are to be dropped elsewhere individually, or use a (paged) arena allocator and move the value into that. Neither approach seems nice to me ...

[davidbarsky]

LGTM, but I think I'd prefer someone from Astral to also approve this—conflict of interest and all.

[MichaReiser]

We now see this issue in ty :) Worst case scenarios are a 9-12s hang where 64 threads block on a single thread dropping old parsed modules (for an operation that takes 60ms to compute)

[MichaReiser]

A somewhat naive alternative would be to use rayon to spawn multiple threads. Certainly not ideal.

[MichaReiser]

Another alternative is to bound how many items are dropped per revision.

[MichaReiser]

I misunderstood this PR. I assumed it's for LRU/eviction policy. But it's mainly about replacing memos.

Can't we special case this case and drop the memo value in place? We already hold the lock.

[Veykril]

If you mean eagerly dropping memos on eviction, no we cannot as we do not know whether another query is using the memo still.

[MichaReiser]

If you mean eagerly dropping memos on eviction, no we cannot as we do not know whether another query is using the memo still.

Can you say more about when another query would still use that memo after execute completed? Specifically, I think we can drop the value (not the memo) in execute after inserting the new memo (we may need to reshuffle some instructions in fetch_hot), because we hold the query's lock.

[MichaReiser]

What I have in mind is something like this https://github.com/salsa-rs/salsa/compare/master...MichaReiser:salsa:drop-replaced-memo-values?expand=1 (this is all codex generated and some safety claims don't hold without making the enclosing function unsafe too)

I'm not saying it's worth pursuing, especially if we need a different solution for LRU anyway. But I do like the idea of eagerly eviction to reduce peak memory usage and distributing work load across multiple threads.

[Veykril]

Ah I misunderstood you, I thought you were talking about LRU/eviction 😅 Yea I think eager dropping on re-execution should be fine.