fix: preserve complete CI test output and exit status - #1327
Conversation
Save streaming test-on-ci output and flush diagnostics before returning a failure. Retain test logs in generated standalone workflows as artifacts. Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly improves the reliability of CI test diagnostics by ensuring that complete test output and command exit statuses are always preserved. It introduces local logging for Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. CI logs now flow, Failures caught, exit codes known, No more lost output. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request updates the 'wb test-on-ci' command to stream raw output and overwrite '.wb/test-ci.log' on each run. It introduces a 'streamOutput' option in 'startVerificationOutput' to allow streaming output directly while still logging to a file. Additionally, the self-contained test workflow generator in 'wbfy' is updated to run tests in a wrapper that pipes output to a log file and uploads it as a GitHub Actions artifact. Relevant unit tests are added and updated to verify these changes. There are no review comments, so I have no feedback to provide.
Capture workflow logs only for opted-in repositories, propagate tee failures, and label artifacts by platform. Preserve stdin for captured E2E commands. Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Keep inherited stdin for pooled commands as well as E2E execution. Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Retain local diagnostics independently of optional workflow artifacts and record that incomplete logs fail otherwise successful capture. Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Why
WillBooster/review-booster#329 needed a local CI shell wrapper to retain failure diagnostics. Buffered test output followed by immediate process exit can lose data when stdout is piped; central capture avoids repeating that workaround.
Requirements
Customer Summary
CI tests retain their complete output locally while continuing to stream diagnostics. Generated standalone workflows can retain test logs as downloadable artifacts after failures when the repository opts in for secret-free output.
Technical Summary
testOnCi.tshandles command failures after output capture has flushed, preserving the exit status. The verification reporter's optional streaming mode writes output to disk as it arrives and forwards it to the console.selfContainedWorkflow.tscaptures the test command with tee, reads its original pipeline exit code, and uploads the log using the OS, job check-run ID and run attempt when UPLOAD_TEST_LOG=true. A failed log write fails an opted-in run while preserving any nonzero test code. The tee subprocess ignores file-size-limit signals so it can keep draining output after a log failure. Local capture handles short writes and reports write, tail-read and close errors after the command finishes. Quiet verification falls back to console output after a log-write failure so subsequent test diagnostics remain visible. Captured E2E commands inherit stdin so EOF remains observable. Streaming subprocesses skip accumulating a second in-memory copy of output; other spawn callers retain the existing collection default. Forwarded child streams honor stdout/stderr backpressure by pausing until the destination drains. Captured commands preserve terminal colors while respecting explicit color preferences. The README documents local log errors and console fallback for all verification commands.Testing
npm_config_prefer_offline=true bun run verify-full: passed in 1m 22s. A prior unmodified registry-backed test timed out against npm.flatt.tech; cached npm metadata avoided the proxy outage without changing registry configuration.bun run verify; GitHub CI and review-booster completed on the final head.Notes
Reusable workflow capture and failure-time uploads are handled by WillBooster/reusable-workflows#518. Generated workflow changes apply on the next wbfy run.