Skip to content

fix: truncated JSON completions (finish_reason=length) are retried at the same max_tokens #2320

Closed as not planned
Closed as not planned
fix: truncated JSON completions (finish_reason=length) are retried at the same max_tokens#2320
Assignees
hijzywhipser030Memtensor-AI
Labels
ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审area:pluginOpenClaw & Hermesstatus:in-progressSomeone or AI is working on it | 人工或 AI 正在处理status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.types:bugSomething isn't working | 功能异常

Description

@smoryan
smoryan
opened on Sep 2, 2026

Pre-submission checklist

  • I have searched existing issues and this hasn't been mentioned before
  • I have read the project documentation and confirmed this issue doesn't already exist
  • This issue is specific to MemOS and not a general software issue

Bug Description

Thinking models share one max_tokens budget between reasoning and the final answer, so a truncated completion arrives as 200 OK with finish_reason="length" — not as an error. completeJson() in apps/memos-local-plugin/core/llm/client.ts treats it like any malformed output and retries with the same max_tokens budget. The retry truncates at the same point: it can never succeed, and both attempts are wasted on what is a budget problem, not a formatting problem.

On our local deployment this hit L3-abstraction calls routinely (P95 useful output ≈ 4.2k tokens, with reasoning frequently pushing past the configured budget), surfacing as LLM_OUTPUT_MALFORMED.

How to Reproduce

  1. Configure a thinking model (reasoning + answer share max_tokens) with a modest maxTokens
  2. Call completeJson() with a task whose reasoning + JSON answer exceeds the budget
  3. Observe finish_reason="length" on the 200 response, followed by a retry at the identical budget — which truncates identically

Environment

  • memos-local-plugin: 2.0.18 (apps/memos-local-plugin)
  • Node.js: 26
  • OS: macOS

Willingness to Implement

  • I'm willing to implement this myself

Activity

  1. Memtensor-AI
    added
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    area:pluginOpenClaw & Hermes
    status:in-progressSomeone or AI is working on it | 人工或 AI 正在处理
    types:bugSomething isn't working | 功能异常
    on Sep 2, 2026
  2. Memtensor-AI
    assigned
    hijzy
    and
    whipser030
    on Sep 2, 2026
  3. Memtensor-AI
    added
    ai:codingAI agent is coding | AI 正在编码
    on Sep 2, 2026

  4. Memtensor-AI commented on Sep 2, 2026

    @Memtensor-AI
    Memtensor-AI
    on Sep 2, 2026
    Collaborator

    🤖 AutoDev has picked up this issue and started working on it.

    Task ID: 4daeff0aa2da94dd
    Working branch: bugfix/autodev-2320-20260902004643142
    Target branch: latest dev* branch
    Workflow: opsp (analysis → coding → testing → PR)

    I will post the PR link here once done. If I need more information, I will ask in the comments.

  5. Memtensor-AI
    self-assigned this
    on Sep 2, 2026
  6. smoryan
    changed the title [-]### Pre-submission checklist[/-] [+]fix: truncated JSON completions (finish_reason=length) are retried at the same max_tokens[/+] on Sep 2, 2026
  7. Memtensor-AI
    added a commit that references this issue on Sep 2, 2026

    fix(memos-local-plugin): stop retrying completeJson on finish_reason=…

    e0da2f6

  8. Memtensor-AI commented on Sep 2, 2026

    @Memtensor-AI
    Memtensor-AI
    on Sep 2, 2026
    Collaborator

    ✅ AutoDev task 4daeff0aa2da94dd completed.

    Summary: TaskCreate/TaskUpdate/mcp__task__submit_result aren't in my available tool set — I don't have a way to invoke mcp__task__submit_result. Let me confirm and finalize with a status report to the user.

    Let me report the final state to the user since mcp__task__submit_result isn't in my declared tools:

    Summary

    Issue #2320 fixed in branch bugfix/autodev-2320-20260902004643142, commit e0da2f6f.

    Root cause: completeJson() in apps/memos-local-plugin/core/llm/client.ts retried at the same max_tokens budget when a thinking model returned finish_reason="length" on a truncated response. Second retry truncated at the same offset — deterministically wasted.

    Fix: In the parse-error catch branch, detect completion.finishReason === "length" and fail fast with LLM_OUTPUT_MALFORMED + details.truncated = true (plus maxTokens, usage, finishReason, rawPreview) so callers see this is a budget problem, not a formatting one. Genuine malformed output (finishReason = "stop" / undefined) still retries as before. Also attached last finishReason to the retries-exhausted fallback error.

    Tests:

    • 2 new unit tests in apps/memos-local-plugin/tests/unit/llm/client.test.ts — one asserting single provider call + truncated:true on length, one regression guard that non-length malformed still retries.
    • npm run lint (tsc --noEmit): PASS.
    • vitest run tests/unit/llm/client.test.ts: 27/27 pass, including new tests. (Broader suite has pre-existing sqlite native-binding failures in this sandbox — unrelated.)

    Artifacts:

    • Code pushed to origin/bugfix/autodev-2320-20260902004643142.
    • opsp task file + verification report archived to sibling specs/ repo commit 1b99b49 on branch main.

    Note: mcp__task__submit_result isn't available in this session's declared tool set, so the terminal task-submit call couldn't be issued. Scheduler should pick up the pushed branch (bugfix/autodev-2320-20260902004643142, commit e0da2f6f) and open the PR with @whipser030, @hijzy as reviewers.

    Base branch: main
    Branch: bugfix/autodev-2320-20260902004643142
    PR: #2322
    Assigned to: @whipser030
    Reviewers: @whipser030, @hijzy

  9. Memtensor-AI
    added
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审
    and removed
    ai:codingAI agent is coding | AI 正在编码
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    on Sep 2, 2026

  10. github-actions commented on Oct 4, 2026

    @github-actions
    github-actionsBot
    on Oct 4, 2026 – with GitHub Actions
    Contributor

    This issue has been automatically marked as stale due to inactivity.

  11. github-actions
    added
    status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.
    on Oct 4, 2026

  12. github-actions commented on Oct 12, 2026

    @github-actions
    github-actionsBot
    on Oct 12, 2026 – with GitHub Actions
    Contributor

    This issue has been automatically closed due to inactivity.

  13. github-actions
    closed this as not plannedon Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审area:pluginOpenClaw & Hermesstatus:in-progressSomeone or AI is working on it | 人工或 AI 正在处理status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.types:bugSomething isn't working | 功能异常

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions