[CI] Fix mxfp8 TrtllmGenMoe test#23125
Conversation
|
Hi @Fridge003 @b8zhong could you help review this tiny ci fix? Thank you! |
There was a problem hiding this comment.
Code Review
This pull request updates the test configuration for the Flashinfer TRTLLM MoE backend by switching to a specific MXFP8 quantized model and changing the --fp8-gemm-backend to flashinfer_cutlass. Feedback suggests moving the model checkpoint from a personal repository to an official organization account to ensure long-term availability and security for the CI suite. Additionally, there is a concern that switching the GEMM backend to cutlass in a test specifically intended for flashinfer_trtllm reduces test coverage for the target integration, suggesting that the underlying numerical issues should be fixed rather than bypassed.
|
/rerun-stage stage-c-test-4-gpu-b200 |
|
/rerun-failed-ci |
4 similar comments
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
|
/rerun-failed-ci |
Motivation
@HumansAnd
mxfp8 test regressed after #21667 (reference: #22136).
Modifications
flashinfer_cutlassinstead offlashinfer_trtllmfor fp8 gemm backend, sinceflashinfer_trtllmseems to require some padding fix for stable numerics, which is missing in current implementation.Accuracy Tests
Speed Tests and Profiling
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ci