perf(multimodal): optimize SHM input transport

[yechank-nvidia]

Summary

• Report the loaded vision encoder dtype so SMG can serialize multimodal tensors in the destination dtype.
• Interleave large SHM-to-pinned copies with asynchronous H2D transfers.
• Overlap SHM unlink cleanup with H2D for single large tensors.
• For large tensor-parallel inputs, copy SHM on one vision TP rank and broadcast the GPU tensor instead of repeating the host copy on every rank.
• Copy single TP source tensors directly from SHM into the destination GPU buffer.
• Preserve the existing staged path for small, mixed-dtype, inline, and non-CUDA inputs.

Test Plan

• Verified vision encoder dtype discovery for direct and nested vision towers.
• Verified SHM copy, pinned-memory staging, direct destination copy, cleanup, and dtype validation.
• Verified TP broadcast selection thresholds and fallback behavior for mixed dtype and inline tensors.
• Verified CUDA value preservation for both small staged tensors and large interleaved tensors.
• Benchmarked SHM/H2D paths on B200 with single-GPU and TP configurations.
• pytest -q test/runtime/test_multimodal_encoder_dtype.py test/runtime/test_multimodal_shm_transport.py test/runtime/test_multimodal_tp_broadcast.py
• pre-commit run --all-files

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1d749e519b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7afff1105d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Report encoder dtype from encode workers

When server_args.disaggregation_mode == "encode", run_event_loop returns through run_encode_loop before this ready payload is sent, and the encode loop’s own ready dict still only includes the token/sequence fields. In that encode-worker deployment, get_server_info() therefore has no multimodal_encoder_dtype, so SMG cannot serialize multimodal tensors in the destination vision dtype for the path that actually loads only the vision tower; mirror this field in the encode-loop ready message as well.

Useful? React with 👍 / 👎.