Skip to content

Updating gpt-oss-20b to use mxfp8 - #6

Open
chelsea0x3b wants to merge 1 commit into
lambda-moe-tflopsfrom
gpt-oss-mxfp8
Open

Updating gpt-oss-20b to use mxfp8#6
chelsea0x3b wants to merge 1 commit into
lambda-moe-tflopsfrom
gpt-oss-mxfp8

Conversation

@chelsea0x3b

Copy link
Copy Markdown
Collaborator

Loss dynamics

step:   1 loss: 12.81606 grad_norm: 13.6420 memory: 149.41GiB(83.78%) tps:    705 tflops:  22.08
step:  10 loss: 10.25156 grad_norm:  4.9128 memory: 169.74GiB(95.17%) tps: 12,659 tflops: 396.40
step:  20 loss:  8.10361 grad_norm:  5.8574 memory: 169.74GiB(95.17%) tps: 19,750 tflops: 618.45
step:  30 loss:  7.61133 grad_norm:  1.7541 memory: 169.74GiB(95.17%) tps: 19,688 tflops: 616.50
step:  40 loss:  7.22928 grad_norm:  1.5464 memory: 169.74GiB(95.17%) tps: 19,617 tflops: 614.26
step:  50 loss:  7.01949 grad_norm:  1.6498 memory: 169.74GiB(95.17%) tps: 19,640 tflops: 614.99
step:  60 loss:  6.84448 grad_norm:  3.0737 memory: 169.74GiB(95.17%) tps: 19,692 tflops: 616.64
step:  70 loss:  6.72140 grad_norm:  1.7173 memory: 169.74GiB(95.17%) tps: 19,709 tflops: 617.17
step:  80 loss:  6.52672 grad_norm:  1.5986 memory: 169.74GiB(95.17%) tps: 19,681 tflops: 616.28
step:  90 loss:  6.48507 grad_norm:  1.5715 memory: 169.74GiB(95.17%) tps: 19,707 tflops: 617.09
step: 100 loss:  6.44737 grad_norm:  4.9927 memory: 169.74GiB(95.17%) tps: 19,775 tflops: 619.22

Full log here:
gpt-oss-mxfp8.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant