-
Notifications
You must be signed in to change notification settings - Fork 10
All issues
Issue creation is restricted in this repository
Issues
is:issue state:open
is:issue state:open
Search results
sb-cli cloud evaluation marks all SWE-bench Lite test submissions as failed, while local official eval resolves same predictions
documentationImprovements or additions to documentationImprovements or additions to documentationStatus: Open.#31 In SWE-bench/sb-cli;- Status: Open.#28 In SWE-bench/sb-cli;
need pull access to the Docker image: swebench/sweb.eval.x86_64.django_1776_django-12406:latest
documentationImprovements or additions to documentationImprovements or additions to documentationStatus: Open.#34 In SWE-bench/sb-cli;- Status: Open.#27 In SWE-bench/sb-cli;
- Status: Open.#26 In SWE-bench/sb-cli;
[Bug] Evaluation results dropped to 0% despite successful submission and valid predictions
bugSomething isn't workingSomething isn't workingStatus: Open.#32 In SWE-bench/sb-cli;- Status: Open.#25 In SWE-bench/sb-cli;
No such image: swebench/sweb.eval.x86_64... when executing swebench.harness.run_evaluation
bugSomething isn't workingSomething isn't workingStatus: Open.#35 In SWE-bench/sb-cli;swe-bench-verified disappeared from sb-cli ?
bugSomething isn't workingSomething isn't workingStatus: Open.#33 In SWE-bench/sb-cli;sb-cli false negatives for tests that are skipped for one-core machines
bugSomething isn't workingSomething isn't workingStatus: Open.#29 In SWE-bench/sb-cli;- Status: Open.#22 In SWE-bench/sb-cli;
- Status: Open.#20 In SWE-bench/sb-cli;