From NVIDIA Megatron-LM for visibility by RaymondLi0 · Pull Request #18 · bigcode-project/Megatron-LM

RaymondLi0 · 2023-01-24T20:01:13Z

No description provided.

…dev (#4299) Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

Signed-off-by: Cory Ye <[email protected]>

…failures (#4209) Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

…e tests (#4238) Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

Signed-off-by: dimapihtar <[email protected]>

…er / torchrun) (#4298) Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

Signed-off-by: Maanu Grover <[email protected]> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

…oad (#4267) Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

…ude/skills → skills/ (#4303) Signed-off-by: oliver könig <[email protected]>

…and AGENTS.md → CLAUDE.md

…al tensor (#4265)

Signed-off-by: dimapihtar <[email protected]>

Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

Signed-off-by: oliver könig <[email protected]> Signed-off-by: Oliver Koenig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

) Co-authored-by: liaoyang <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Deepak Narayanan <[email protected]>

Signed-off-by: oliver könig <[email protected]>

…4711) Co-authored-by: peibli <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]>

…24) → main (#4636) Signed-off-by: Kirthi Shankar Sivamani <[email protected]> Signed-off-by: ksivamani <[email protected]> Signed-off-by: Xin Yao <[email protected]> Signed-off-by: Zhongbo Zhu <[email protected]> Co-authored-by: Kirthi Shankar Sivamani <[email protected]> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Xin Yao <[email protected]> Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> Co-authored-by: Gao Deng <[email protected]> Co-authored-by: Gao Deng <[email protected]> Co-authored-by: Zhongbo Zhu <[email protected]>

Signed-off-by: Pavel Gein <[email protected]> Co-authored-by: Antoni-Joan Solergibert <[email protected]>

…an (#4695) Co-authored-by: Gao Deng <[email protected]>

Signed-off-by: dimapihtar <[email protected]> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

Signed-off-by: Xiaowei Ren <[email protected]> Co-authored-by: Philip Petrakian <[email protected]>

Co-authored-by: Gao Deng <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Philip Petrakian <[email protected]>

…on impl, inference scope, and per-layer capture modules (#4292) Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

Signed-off-by: dimapihtar <[email protected]>

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> Co-authored-by: Charlie Truong <[email protected]>

…form PP (#4775) Co-authored-by: Abhishree Thittenamane <[email protected]> Co-authored-by: root <[email protected]>

Signed-off-by: Keval Morabia <[email protected]>

#4806) Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

Signed-off-by: Charlie Truong <[email protected]>

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12

RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12

ko3n1g force-pushed the main branch from ad5a627 to cd71d4c Compare February 18, 2026 09:18

ko3n1g and others added 27 commits April 14, 2026 17:27

ci: mark test_fused_indexer_loss_gradient_tp_consistency as flaky_in_…

3d7a701

…dev (#4299) Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

Fix typo in PR4133. (#4277)

4cef23c

Signed-off-by: Cory Ye <[email protected]>

ci: add retry loop to apt-get update to handle transient mirror sync …

bcec618

…failures (#4209) Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

fix: enforce correct pass thresholds for deterministic and approximat…

4e85d74

…e tests (#4238) Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

remove legacy biencoder and realm models (#4205)

d245c44

Signed-off-by: dimapihtar <[email protected]>

ci: add configurable launcher support for functional tests (ft_launch…

6636eb0

…er / torchrun) (#4298) Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

chore: document --target main for local Docker builds (#4307)

d530a04

Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

Extract args init to launch scripts (#4225)

97aca2f

Signed-off-by: Maanu Grover <[email protected]> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

[Main] Fix TE version check for retain_pinned_cpu_buffers in cpu offl…

c2d1a8f

…oad (#4267) Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

chore: rotate oncall schedule

b342602

Fix documented shape (#3486)

1d344ae

ci: add sync-skills workflow, rename CLAUDE.md → AGENTS.md, move .cla…

4a79536

…ude/skills → skills/ (#4303) Signed-off-by: oliver könig <[email protected]>

chore(beep boop 🤖): symlink skills/ → .claude/skills, .agents/skills …

57fc3ae

…and AGENTS.md → CLAUDE.md

Get device correctly when module returns a dict instead of individu…

23265d2

…al tensor (#4265)

remove vision legacy code (#4202)

e69cf43

Signed-off-by: dimapihtar <[email protected]>

feat: long convergence resiliency for release tests (#4335)

f098fe8

Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

ci(action): improve GitHub Actions output UX (#4337)

8681ebb

Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

build: bump TransformerEngine to release_v2.14 (#4331)

ceac269

Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

feat: add create-issue skill (#4338)

efbe7a1

Signed-off-by: oliver könig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

Set megatron-fsdp to 0.5.0

97f9ab6

M4 leftover for TE cuda graph (#3137)

01eb7e8

fix: wait for async P2P send before deallocating output tensor (#4047)

260cba7

ci(gb200): add 1-node mr-github functional test variants (#4334)

2aeaf56

Signed-off-by: oliver könig <[email protected]> Signed-off-by: Oliver Koenig <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>

Fix potential coredump issue that occurs when saving a checkpoint (#1871

ded22f4

) Co-authored-by: liaoyang <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Deepak Narayanan <[email protected]>

docs: bump versions1.json to 0.17.0 (latest) (#4360)

30bc230

Signed-off-by: oliver könig <[email protected]>

Port DeepSeek Sparse Attention to MambaModel (#3553)

a00e944

Add tables and histogram for RL staleness (#4097)

23663a8

sidsingh-nvidia and others added 30 commits May 14, 2026 22:59

Inference: Optimize Prefill Engine Steps for Nemotron (#4764)

9b4074b

Strengthen test_checkpoint to verify distributed checkpoint behavior (#…

493e2be

…4711) Co-authored-by: peibli <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]>

Disable MSC by default; opt in via --enable-msc (#4629)

95bba94

[fix] Use MSC for checking checkpoint existence (#4251)

f2070ee

Signed-off-by: Pavel Gein <[email protected]> Co-authored-by: Antoni-Joan Solergibert <[email protected]>

ci: tolerate git-gc race in /home/runner chown after checkout (#4808)

b3b6719

Reorder mtp_post_process after attention backward in 1F1B schedule pl…

212f885

…an (#4695) Co-authored-by: Gao Deng <[email protected]>

Update copy-pr-bot.yaml [skip ci]

f92a207

additional tests for nvrx (#4522)

f9952cb

Signed-off-by: dimapihtar <[email protected]> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

[Main][feat] Support A2A Overlap for Megatron-FSDP (#3797)

e9d9a4c

add is_torch_min_version in fsdp src (#4812)

7371e01

Signed-off-by: Xiaowei Ren <[email protected]> Co-authored-by: Philip Petrakian <[email protected]>

Add high-priority A2A stream and HybridEP preprocessing SMs (#4694)

54f90af

Co-authored-by: Gao Deng <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Philip Petrakian <[email protected]>

Refactor CUDA graph API: decompose cuda_graph_scope into full_iterati…

e41b370

…on impl, inference scope, and per-layer capture modules (#4292) Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

chore(beep boop 🤖): Bump (main) (2026-05-18)

a1c5f5d

Tokenizers updates (#4780)

ab28544

Signed-off-by: dimapihtar <[email protected]>

Fix no nvrx tests (#4847)

5096459

Signed-off-by: dimapihtar <[email protected]>

Thread custom process groups through MoE grad finalization (#4782)

aa786b7

Fix unit tests (#4689)

648b916

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

Tests/dynamic inference functional coverage (#4761)

6c3fa6d

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]> Co-authored-by: Charlie Truong <[email protected]>

Update copy-pr-bot.yaml [skip ci]

74daec9

Fix oncall references (#4722)

952efb4

Update golden values for nightly functional tests (#4850)

b2a8ec7

fix(inference): size DynamicInferenceContext KV layer_map for non-uni…

43fb2f5

…form PP (#4775) Co-authored-by: Abhishree Thittenamane <[email protected]> Co-authored-by: root <[email protected]>

Modernize post-training modelopt example scripts (#4807)

14aaa7e

Signed-off-by: Keval Morabia <[email protected]>

chore: rotate oncall schedule

38986a9

test: add inference performance test harness for GPT 583M, hybrid 2B,… (

23ba357

#4806) Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

ci: Prevent shell trace in parts of _run_training.sh (#4884)

1ffcd1d

Signed-off-by: Charlie Truong <[email protected]>

Update copy-pr-bot.yaml [skip ci]

5df3e4e

ci: Gate optional CI jobs with repository variables (#4907)

990fa69

Signed-off-by: Charlie Truong <[email protected]>

Fix tokenizers bug in nightly (#4833)

e69a1ea

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

From NVIDIA Megatron-LM for visibility#18

From NVIDIA Megatron-LM for visibility#18
RaymondLi0 wants to merge 6985 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main

RaymondLi0 commented Jan 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

RaymondLi0 commented Jan 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants