Skip to content

fix(activation): keep the stored KV cache read-only during generation (#2301) - #2552

Open
RerankerGuo wants to merge 2 commits into
MemTensor:mainfrom
RerankerGuo:feature/fix-activation-cache-aliasing
Open

RerankerGuo wants to merge 2 commits into
MemTensor:mainfrom
RerankerGuo:feature/fix-activation-cache-aliasing

fix(activation): copy the KV cache only once per generation

55d2d84
Select commit
Loading
Failed to load commit list.
Sign in for the full log view