Skip to content

Add endless-scrolling TUI table for interactive list commands#4729

Closed
simonfaltum wants to merge 46 commits intomainfrom
simonfaltum/list-tui-paginated
Closed

Add endless-scrolling TUI table for interactive list commands#4729
simonfaltum wants to merge 46 commits intomainfrom
simonfaltum/list-tui-paginated

Conversation

@simonfaltum
Copy link
Copy Markdown
Member

@simonfaltum simonfaltum commented Mar 12, 2026

Why

Large list commands are hard to use in a terminal today. They print the full result set before users can do anything, which is slow for big lists and makes it hard to browse results. Interactive terminals should get a faster, more navigable experience without changing scripted output.

Changes

Before: list commands drained the full paginated iterator and rendered everything through text templates or JSON.

Now: when the CLI is running interactively (TTY stdin + stdout + stderr, color enabled), list commands with a registered table configuration open a scrollable TUI table that loads more rows on demand. Piped output and --output json keep the existing behavior. Commands without an explicit table config continue to use template rendering.

Implementation:

  • Extract shared styles, constants, and helpers from tableview.go into common.go
  • Add reusable tableview config, registry, and wrapper types
  • Add a Bubble Tea paginated model with fetch-on-demand, cursor navigation, horizontal scrolling, and optional server-side search
  • Route interactive list rendering through the TUI when a TableConfig is registered, fall through to templates otherwise
  • Integrate with cmdio Tea program lifecycle (acquireTeaProgram/releaseTeaProgram)
  • Switch experimental/aitools to the shared static table renderer

Post-review fixes

  • Fix UTF-8 backspace corruption in search (byte truncation to proper rune handling via utf8.DecodeLastRuneInString)
  • Fix RunPaginated silently dropping model-level fetch errors (now checks FinalModel.Err() like render.go)
  • Recompute column widths on every batch, not just the first
  • Extract restorePreSearchState to eliminate duplicated search-restore logic
  • Show fetch errors in footer while keeping existing rows visible (instead of replacing all data)
  • Fix space key not accepted in search input (handle tea.KeySpace in both paginated and non-paginated tables)
  • Separate loading and searching concerns: loading is purely "fetch in-flight", searching flag blocks maybeFetch during search input mode (fixes stuck pagination and overlapping fetch races)

Simplification pass

  • Add generic tableview.Col[T] / ColMax[T] helpers so each column drops from a 3-line func literal with explicit type assertion to a single-line generic call. Converted all 18 cmd/workspace/*/overrides.go files.
  • Consolidate 9 scattered search fields (searching, searchLoading, searchInput, debounceSeq, hasSearchState, savedRows, savedIter, savedExhaust) into one searchState struct with an optional *savedSearch sub-struct for the pre-search snapshot. restorePreSearchState and executeSearch shrink significantly; the state lifecycle becomes self-documenting.
  • Collapse trivial View / renderFooter / search input key / fetch tests into table-driven form.

Test plan

  • go test ./libs/tableview/... ./libs/cmdio/... ./cmd/root/... ./experimental/aitools/cmd/...
  • Manual smoke test in a TTY: verify the first page renders immediately and more rows load while scrolling
  • Manual smoke test with piped output and --output json: verify existing output is unchanged
  • make checks passes
  • make lintfull passes (0 issues)

@eng-dev-ecosystem-bot
Copy link
Copy Markdown
Collaborator

eng-dev-ecosystem-bot commented Mar 12, 2026

Commit: be88c2e

Run: 23569035345

@simonfaltum simonfaltum force-pushed the simonfaltum/list-tui-paginated branch from 1bf0772 to e672906 Compare March 13, 2026 06:23
@simonfaltum simonfaltum marked this pull request as ready for review March 13, 2026 10:33
Copy link
Copy Markdown
Contributor

@shreyas-goenka shreyas-goenka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This review was posted by Claude (AI assistant). Shreyas will do a separate, more thorough review pass.

Priority: HIGH — Performance and design concerns in TUI table infrastructure

MAJOR: O(n) re-render on every keystroke

The search/filter functionality appears to re-render the entire table on each keystroke. For large result sets, this will cause noticeable lag. Consider debouncing the search input or using incremental filtering.

MAJOR: Partially-consumed iterator after search

When a user searches/filters, the iterator may have already been partially consumed. If the search is cleared, previously consumed items that didn't match won't be re-fetched. This could lead to confusing behavior where clearing a search shows fewer results than expected.

MEDIUM: Entire feature unreachable until follow-up PRs

This is 1535 lines of new code that is unreachable until the override PRs (#4731, #4732) land. Consider whether the infrastructure could be reviewed more effectively alongside at least one consumer.

What looks good

  • Clean separation of concerns between model, view, and update
  • Good use of Bubble Tea framework patterns
  • Table column configuration is flexible and well-designed

@simonfaltum simonfaltum force-pushed the simonfaltum/list-tui-paginated branch from 8fa51c9 to d7a7104 Compare March 13, 2026 12:40
Search input now triggers server-side filtering automatically after the
user stops typing for 200ms, instead of waiting for Enter. This prevents
redundant API calls on each keystroke while keeping the text input
responsive. Enter still executes search immediately, bypassing the
debounce.

Uses Bubble Tea's tick-based message pattern with a sequence counter to
discard stale debounce ticks when the user types additional characters
before the delay expires.
Fix four issues in the paginated TUI:

1. Entering search mode now sets loading=true to prevent maybeFetch from
   starting new fetches against the shared iterator while in search mode.
   In-flight fetches are discarded via the generation check.

2. executeSearch sets loading=true (was false) to prevent overlapping fetch
   commands when a quick scroll triggers maybeFetch before the first search
   fetch returns.

3. Pressing esc to close search now restores savedRows, savedIter, and
   savedExhaust (same as clearing the query via enter with empty input).

4. RenderIterator now checks the final model for application-level errors
   via the new FinalModel interface, since tea.Program.Run() only returns
   framework errors.
@simonfaltum simonfaltum requested review from tejaskochar-db and removed request for tejaskochar-db April 9, 2026 20:35
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 9, 2026

Approval status: pending

/cmd/workspace/apps/ - needs approval

Files: cmd/workspace/apps/overrides.go, cmd/workspace/apps/overrides_test.go
Suggested: @pkosiec
Also eligible: @MarioCadenas, @arsenyinfo, @fjakobs, @jamesbroadhead, @Shridhad, @atilafassina, @keugenek, @igrekun, @pffigueiredo, @ditadi, @calvarjorge

/experimental/aitools/ - needs approval

Files: experimental/aitools/cmd/render.go, experimental/aitools/cmd/render_test.go
Suggested: @pkosiec
Also eligible: @MarioCadenas, @arsenyinfo, @fjakobs, @jamesbroadhead, @Shridhad, @atilafassina, @keugenek, @igrekun, @pffigueiredo, @ditadi, @calvarjorge, @lennartkats-db

/libs/cmdio/ - needs approval

Files: libs/cmdio/capabilities.go, libs/cmdio/render.go
Suggested: @parthban-db
Also eligible: @tanmay-db, @renaudhartert-db, @hectorcast-db, @Divyansh-db, @tejaskochar-db, @mihaimitrea-db, @chrisst, @rauchy

General files (require maintainer)

29 files changed
Based on git history:

  • @pietern -- recent work in cmd/workspace/workspace/, cmd/workspace/jobs/, cmd/workspace/serving-endpoints/

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.

simonfaltum and others added 2 commits April 14, 2026 09:34
Init() returns a tea.Cmd that fetches rows in a goroutine but cannot
modify the model (Bubbletea's Init() only returns a Cmd, not a Model).
Without loading=true in the constructor, a keypress arriving before the
first rowsFetchedMsg would pass maybeFetch's guard and fire a second
concurrent fetch on the same non-thread-safe iterator.

Co-authored-by: Isaac
@simonfaltum simonfaltum requested a review from pietern April 14, 2026 13:16
Copy link
Copy Markdown
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard to do a full review. I ran through the UX in the snapshot build and ran a separate review (see below)

How we do the plumbing is most important to me because this adds a second mode.

Review finding:

renderTableLines and paginatedModel.renderContent use "─" (box-drawing horizontal), but RenderStaticTable uses "-" (ASCII hyphen)

Comment thread cmd/root/io.go Outdated

cmdIO := cmdio.NewIO(ctx, f.output, cmd.InOrStdin(), cmd.OutOrStdout(), cmd.ErrOrStderr(), headerTemplate, template)
ctx = cmdio.InContext(ctx, cmdIO)
ctx = cmdio.WithCommand(ctx, cmd)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing mechanism to pass information from the autogen'd commands (or the overrides) into the command is via the annotations. This adds another mechanism.

I'm not biased to either but we should maintain a single system for associating static info with commands.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Removed WithCommand/CommandFromContext and now thread *cobra.Command as an explicit parameter to RenderIterator. The registry (RegisterConfig/GetConfig) is the only new mechanism, annotations still handle templates.

func listOverride(listCmd *cobra.Command, _ *sql.ListAlertsRequest) {
listCmd.Annotations["template"] = cmdio.Heredoc(`
{{range .}}{{green "%s" .Id}} {{.DisplayName}} {{.State}}
{{end}}`)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need the template if we also use the tableview?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the template is the fallback for non-interactive output (piped, --output text, non-TTY). The TUI only runs when SupportsTUI() is true. Added a comment to all overrides clarifying this.

"github.com/stretchr/testify/require"
)

func TestListTableConfig(t *testing.T) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why test here and not other commands?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apps has nested pointer dereferences (ComputeStatus.State, ActiveDeployment.Status.State) that can nil-panic if the Extract functions aren't careful. The other overrides are single-field type assertions so there's less to go wrong.

Comment thread libs/tableview/autodetect.go Outdated
// AutoDetect creates a TableConfig by reflecting on the element type of the iterator.
// It picks up to maxAutoColumns top-level scalar fields.
// Returns nil if no suitable columns are found.
func AutoDetect[T any](iter listing.Iterator[T]) *TableConfig {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is currently unused.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

if m.cfg.Search != nil {
m.searching = true
m.searchInput = ""
m.viewport.Height--
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this alter the viewport size?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shrinks by one row to make room for the search input bar at the bottom. Added a comment.

Comment thread libs/tableview/paginated.go Outdated
@pietern
Copy link
Copy Markdown
Contributor

pietern commented Apr 14, 2026

Btw, the initial "Fetching results..." and then the experience when waiting for more results in the table are different. I recommend putting the user immediately in the table view and clearly showing a marker that rows are being fetched.

When using a terminal with >50 rows, the initial page loads but doesn't continue loading. I have to first navigate to the last row before it triggers fetching the next set of rows.

Forward slash to enter search doesn't work for me.

simonfaltum and others added 3 commits April 15, 2026 09:24
…mprove consistency

- Remove useless TestNewPaginatedProgramSetsLoadingTrue (linter unusedwrite)
- Make ctrl+c quit from search mode instead of canceling search
- Add comments explaining viewport height adjustments for search bar
- Remove unused AutoDetect function and its tests
- Add comments explaining template+tableview coexistence in all overrides
- Thread cobra.Command through RenderIterator parameter instead of context,
  removing WithCommand/CommandFromContext context mechanism
- Unify separator character to box-drawing (U+2500) in RenderStaticTable

Co-authored-by: Isaac
@simonfaltum
Copy link
Copy Markdown
Member Author

Good points. I'll look into showing the table immediately with a loading indicator instead of the separate "Fetching results..." screen. Will debug the forward slash issue too, might be terminal-specific.

@simonfaltum simonfaltum requested a review from pietern April 15, 2026 09:42
simonfaltum and others added 5 commits April 15, 2026 12:49
- Keep previous rows visible during search (show "Searching..." footer
  instead of clearing rows immediately)
- Clone TableConfig before mutation in disableSearchIfFilterSet
- Remove unused cmd parameter from RenderIterator and update all callers
- Export and deduplicate SanitizeControlWhitespace
- Fix inaccurate highlightSearch comment about case folding

Co-authored-by: Isaac
Each cmd/workspace/*/overrides.go had repeated type assertions like
`v.(jobs.BaseJob).Field` in every column Extract closure. Introduce a
generic `tableview.Col[T]` helper (and `ColMax[T]` for width-capped
columns) that hides the assertion, then convert all 18 override files
to use it.

Each column definition drops from a 3-line func literal with an
explicit type assertion to a single-line generic call, making the
shape of each table at a glance.

Co-authored-by: Isaac
paginatedModel had 9 scattered fields handling server-side search
(searching, searchLoading, searchInput, debounceSeq, hasSearchState,
savedRows, savedIter, savedExhaust, limitReached was left as its own
thing). Many of the recent fixes in this PR's history were races
between these fields getting out of sync.

Group them into a searchState struct with an optional *savedSearch
sub-struct for the pre-search snapshot. restorePreSearchState and
executeSearch become markedly simpler because "there is no saved
state" is one nil check instead of four zeroed fields.

Also consolidate a few trivial table tests (View, renderFooter,
search input keys, fetch) into table-driven form.

Co-authored-by: Isaac
@simonfaltum simonfaltum force-pushed the simonfaltum/list-tui-paginated branch from ba1b90a to e269ea3 Compare April 17, 2026 13:43
simonfaltum added a commit that referenced this pull request Apr 17, 2026
When stdin and stdout are both TTYs and a TableConfig is registered,
list commands now stream their first page of rows to stdout and prompt:

  [space] more  [enter] all  [q|esc] quit

Piped output and `--output json` keep the existing behavior.

Alternative approach to #4729: instead of a full Bubble Tea TUI with
scrolling, search, and cursor navigation, this renders a plain text
table page-by-page and reads a single key in raw terminal mode to
drive paging. Same TableConfig / Col[T] infrastructure, minus the
interactive browser.

Wiring:
- `libs/tableview/config.go`, `context.go`, `wrap.go`: TableConfig,
  RowIterator, Col[T]/ColMax[T] generic helpers, cobra PreRunE wiring.
- `libs/tableview/common.go`: extracted `computeColumnWidths` and
  `renderTableToLines` so they can be shared between the existing
  static aitools browser and the new pager.
- `libs/tableview/simple.go`: the pager itself (raw-mode key reader,
  fetch-on-demand, per-column MaxWidth truncation).
- `libs/cmdio/render.go`: `RenderIterator` routes to the pager when
  `Capabilities.SupportsPager()` returns true.
- `cmd/workspace/*/overrides.go`: TableConfigs for the 18 most
  common list commands.

Test plan:
- `go test ./libs/tableview/... ./libs/cmdio/... ./cmd/workspace/...`
- `make checks` passes
- `make lintfull` passes (0 issues)
- Manual smoke in a TTY: space pages through, enter drains, q/esc exits.

Co-authored-by: Isaac
@simonfaltum
Copy link
Copy Markdown
Member Author

Decided to go with another approach that solves pagination initially. Can always re-open to extend this functionality if needed.

@simonfaltum simonfaltum deleted the simonfaltum/list-tui-paginated branch April 21, 2026 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants