[benchmark_inference.py] Specify tp_size to StaticCache#2784

crcrpar wants to merge 3 commits into

mainLightning-AI/lightning-thunder:mainfrom

mkozuki/inference_bench_tensor_parallel_sharding_fixLightning-AI/lightning-thunder:mkozuki/inference_bench_tensor_parallel_sharding_fixCopy head branch name to clipboard