Skip to content

Add reproduction for inconsistent door-opening actions - #87

Merged
MarcCote merged 4 commits into
mainfrom
copilot/reproduce-door-opening-inconsistency
Aug 20, 2026
Merged

Add reproduction for inconsistent door-opening actions#87
MarcCote merged 4 commits into
mainfrom
copilot/reproduce-door-opening-inconsistency

Conversation

Copilot AI commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

ScienceWorld generates inconsistent commands for opening doors: the bedroom uses open bedroom door, while most locations use open door to <location>.

  • Reproduction
    • Add scripts/reproduce_issue_82.py.
    • Load deterministic task 1-1 and print valid door-opening actions.
    • Ensure the simulator process closes after execution.
open bedroom door
open door to greenhouse
open door to kitchen

Co-authored-by: MarcCote <660004+MarcCote@users.noreply.github.com>
Copilot AI changed the title [WIP] Add script to reproduce door opening inconsistency Add reproduction for inconsistent door-opening actions Aug 4, 2026
Copilot AI requested a review from MarcCote August 4, 2026 17:14
@MarcCote

Copy link
Copy Markdown
Collaborator

@copilot check that recent changes in main fixes this issue.

@MarcCote
MarcCote marked this pull request as ready for review August 19, 2026 14:42

Copilot AI commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

@copilot check that recent changes in main fixes this issue.

Checked main at 0ff14f9; it does not fix #82. The reproducer still emits both open bedroom door/open art studio door and open door to <location> forms. The recent changes make enumeration deterministic but do not normalize portal commands.

Copilot AI commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

@copilot check that recent changes in main fixes this issue.

No—this still reproduces on latest origin/main (0ff14f9). It emits both open bedroom door / open art studio door and open door to greenhouse / kitchen / living room / workshop, identical to this branch.

@MarcCote

Copy link
Copy Markdown
Collaborator

@copilot let's propose a fix for this then. ideally, without introducing breaking changes.

Copilot AI and others added 2 commits August 19, 2026 14:51
…r-opening-inconsistency

Co-authored-by: MarcCote <660004+MarcCote@users.noreply.github.com>
Co-authored-by: MarcCote <660004+MarcCote@users.noreply.github.com>

Copilot AI commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

@copilot let's propose a fix for this then. ideally, without introducing breaking changes.

Implemented in b57cca1. Generated portal actions now consistently use door to <location>, while parser aliases such as open bedroom door remain accepted. Added regression coverage for canonical output and both legacy/canonical commands.

return this.name + " to " + connectsToContainer.get.name
}

return this.name + " from " + connectsFrom.name + " to " + connectsTo.name

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot when would that be used as a canonical referent? Give concrete example from playing ScienceWorld.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no player-facing example. With easy/teleportAction, the all-universe referent LUT computes this fallback for remote doors (for example, door from bathroom to kitchen while the player is in the hallway), but teleport generation accepts only locations. Open/close/move actions use visible doors and therefore always use door to <destination>.

return this.name + " to " + connectsToContainer.get.name
}

return this.name + " from " + connectsFrom.name + " to " + connectsTo.name

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot when would that be used as a canonical referent? Give concrete example from playing ScienceWorld.

Copilot AI commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

@copilot check that recent changes in main fixes this issue.

No. Current main at 072eb1c still emits open art studio door and open bedroom door alongside open door to <location> forms. The branch emits only open door to <location> forms.

@copilot let's propose a fix for this then. ideally, without introducing breaking changes.

Implemented in b57cca1. Generated door actions consistently use door to <location>, while legacy inputs such as open bedroom door remain accepted to avoid breaking existing agents.

@MarcCote
MarcCote merged commit e8216d6 into main Aug 20, 2026
35 checks passed
@MarcCote
MarcCote deleted the copilot/reproduce-door-opening-inconsistency branch August 20, 2026 00:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inconsistency of the action of open door to someplace.

2 participants