Add CPU support in fbgemm for FloatToFP8RowwiseQuantized and FP8RowwiseQuantizedToFloat (#5644)#5644
Open
djjatmeta wants to merge 1 commit intopytorch:mainfrom
Open
Add CPU support in fbgemm for FloatToFP8RowwiseQuantized and FP8RowwiseQuantizedToFloat (#5644)#5644djjatmeta wants to merge 1 commit intopytorch:mainfrom
djjatmeta wants to merge 1 commit intopytorch:mainfrom
Conversation
meta-codesync
Bot
changed the title
djjatmeta
force-pushed
the
export-D100724285
branch
from
a042d39 to
316d9fd
Compare
djjatmeta
added a commit
to djjatmeta/FBGEMM-1
that referenced
this pull request
Apr 17, 2026
…seQuantizedToFloat (pytorch#5644) Summary: Add fp8 support on CPU for fbgemm::FloatToFP8RowwiseQuantized GLOBE eager accuracy test - `output_columns = ncols - 2 * sizeof(float)` equals `ncols_aligned` (the full aligned width, NOT the original K). This matches GPU kernel behavior (`quantize_fp8_rowwise.cu:170`). - `std::abs` + `std::max` reduction is equivalent to `MAX(max_elem, -min_elem)` from MTIA ref kernel line 74 - `at::empty` for output (not `at::zeros`) — padding bytes `[K, K_aligned)` left uninitialized, matching GPU (`quantize_fp8_rowwise.cu:223`) and MTIA kernel behavior - Empty tensor early-return with `at::zeros` matches GPU (line 217-221) - Scale zero-pad initialized to 0.0f for PT2 compliance (matches GPU line 52) Differential Revision: D100724285
djjatmeta
added a commit
to djjatmeta/FBGEMM-1
that referenced
this pull request
Apr 17, 2026
…seQuantizedToFloat (pytorch#5644) Summary: Add fp8 support on CPU for fbgemm::FloatToFP8RowwiseQuantized GLOBE eager accuracy test - `output_columns = ncols - 2 * sizeof(float)` equals `ncols_aligned` (the full aligned width, NOT the original K). This matches GPU kernel behavior (`quantize_fp8_rowwise.cu:170`). - `std::abs` + `std::max` reduction is equivalent to `MAX(max_elem, -min_elem)` from MTIA ref kernel line 74 - `at::empty` for output (not `at::zeros`) — padding bytes `[K, K_aligned)` left uninitialized, matching GPU (`quantize_fp8_rowwise.cu:223`) and MTIA kernel behavior - Empty tensor early-return with `at::zeros` matches GPU (line 217-221) - Scale zero-pad initialized to 0.0f for PT2 compliance (matches GPU line 52) Differential Revision: D100724285
djjatmeta
force-pushed
the
export-D100724285
branch
from
316d9fd to
a115518
Compare
djjatmeta
added a commit
to djjatmeta/FBGEMM-1
that referenced
this pull request
Apr 17, 2026
…seQuantizedToFloat (pytorch#5644) Summary: Pull Request resolved: pytorch#5644 Add fp8 support on CPU for fbgemm::FloatToFP8RowwiseQuantized GLOBE eager accuracy test - `output_columns = ncols - 2 * sizeof(float)` equals `ncols_aligned` (the full aligned width, NOT the original K). This matches GPU kernel behavior (`quantize_fp8_rowwise.cu:170`). - `std::abs` + `std::max` reduction is equivalent to `MAX(max_elem, -min_elem)` from MTIA ref kernel line 74 - `at::empty` for output (not `at::zeros`) — padding bytes `[K, K_aligned)` left uninitialized, matching GPU (`quantize_fp8_rowwise.cu:223`) and MTIA kernel behavior - Empty tensor early-return with `at::zeros` matches GPU (line 217-221) - Scale zero-pad initialized to 0.0f for PT2 compliance (matches GPU line 52) Differential Revision: D100724285
djjatmeta
force-pushed
the
export-D100724285
branch
from
a115518 to
b4fa180
Compare
Contributor
|
@djjatmeta has exported this pull request. If you are a Meta employee, you can view the originating Diff in D100724285. |
djjatmeta
added a commit
to djjatmeta/FBGEMM-1
that referenced
this pull request
Apr 17, 2026
…seQuantizedToFloat (pytorch#5644) Summary: X-link: https://github.com/facebookresearch/FBGEMM/pull/2592 Pull Request resolved: pytorch#5644 Add fp8 support on CPU for fbgemm::FloatToFP8RowwiseQuantized GLOBE eager accuracy test - `output_columns = ncols - 2 * sizeof(float)` equals `ncols_aligned` (the full aligned width, NOT the original K). This matches GPU kernel behavior (`quantize_fp8_rowwise.cu:170`). - `std::abs` + `std::max` reduction is equivalent to `MAX(max_elem, -min_elem)` from MTIA ref kernel line 74 - `at::empty` for output (not `at::zeros`) — padding bytes `[K, K_aligned)` left uninitialized, matching GPU (`quantize_fp8_rowwise.cu:223`) and MTIA kernel behavior - Empty tensor early-return with `at::zeros` matches GPU (line 217-221) - Scale zero-pad initialized to 0.0f for PT2 compliance (matches GPU line 52) Differential Revision: D100724285
djjatmeta
force-pushed
the
export-D100724285
branch
from
b4fa180 to
368afba
Compare
…seQuantizedToFloat (pytorch#5644) Summary: X-link: facebookresearch/FBGEMM#2592 Add fp8 support on CPU for fbgemm::FloatToFP8RowwiseQuantized GLOBE eager accuracy test - `output_columns = ncols - 2 * sizeof(float)` equals `ncols_aligned` (the full aligned width, NOT the original K). This matches GPU kernel behavior (`quantize_fp8_rowwise.cu:170`). - `std::abs` + `std::max` reduction is equivalent to `MAX(max_elem, -min_elem)` from MTIA ref kernel line 74 - `at::empty` for output (not `at::zeros`) — padding bytes `[K, K_aligned)` left uninitialized, matching GPU (`quantize_fp8_rowwise.cu:223`) and MTIA kernel behavior - Empty tensor early-return with `at::zeros` matches GPU (line 217-221) - Scale zero-pad initialized to 0.0f for PT2 compliance (matches GPU line 52) Differential Revision: D100724285
djjatmeta
force-pushed
the
export-D100724285
branch
from
368afba to
6508869
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
X-link: https://github.com/facebookresearch/FBGEMM/pull/2592
Add fp8 support on CPU for fbgemm::FloatToFP8RowwiseQuantized GLOBE eager accuracy test
output_columns = ncols - 2 * sizeof(float)equalsncols_aligned(the full aligned width, NOT the original K). This matches GPU kernel behavior (quantize_fp8_rowwise.cu:170).std::abs+std::maxreduction is equivalent toMAX(max_elem, -min_elem)from MTIA ref kernel line 74at::emptyfor output (notat::zeros) — padding bytes[K, K_aligned)left uninitialized, matching GPU (quantize_fp8_rowwise.cu:223) and MTIA kernel behaviorat::zerosmatches GPU (line 217-221)Differential Revision: D100724285