perf: let minimal pages without jQuery widgets skip loading jQuery (opt-in)

[maartenbreddels]

jQuery moves out of the core bundle into a new jquery frontend feature. This cuts about 29 KB gzip from the core on Vue 2 and Vue 3, in production builds. full still preloads it. In minimal, widgets that use jQuery (anywidget widgets, solara.FigurePlotly with plotly 6 or later, pythreejs) need +jquery. This PR is the top of a stack: it sits on the lumino PR (#1240), which sits on #1236 (the output renderers), which sits on #1235.

Problem

Every Solara page downloads and runs one core bundle before it shows the first widget.
With the lumino PR (#1240), this core is 171 KB gzip on Vue 3 and 289 KB on Vue 2 (ipywidgets 8, production build).
About 29 KB of it is jQuery.
Vue widgets never call jQuery methods.
@jupyter-widgets/base wraps the element of each view with $(el), and Vue views only read the element back.
Only the ipywidgets controls, the Output widget, and widgets that call jQuery methods on view.$el (for example anywidget widgets and pythreejs) need jQuery.
A page with only Vue widgets still pays to download and run jQuery.
An earlier CPU profile of the earlier #1236 builds gave estimates for jQuery.
It runs for about 3 ms at normal CPU speed and about 15 ms at 4x CPU slowdown.
I did not repeat that profile on these branches.
The download of 29 KB takes about 23 ms at 10 Mbit/s and about 145 ms at 1.6 Mbit/s (arithmetic, not a measurement).
Slow phones and laptops pay the most.

Change

• A new frontend feature, jquery, holds jQuery.
  The full preset (the default) preloads it, so a full page works as in feat: load pages faster by splitting the frontend into optional features #1235.
  The minimal preset leaves it out, and +jquery turns it on.
• jupyter-controls and output-widget need jquery, because their code uses $.
  On ipywidgets 7 and 8 these are Box, Controller, SelectionContainer and the Output widget, and on ipywidgets 7 also jQuery UI's slider.
  The server adds jquery when you turn one of them on, and full,-jquery is an error while one of them is on.
  In the browser, their chunks load the jquery chunk too, and run only after it ran, also in minimal.
• A widget's own use of jQuery never loads the jquery chunk.
  A widget uses jQuery without asking for it first, so the page cannot know in time which widget needs it.
  The page does not guess.
  Only full, +jquery, the controls and the Output widget load the chunk.
• Every jquery import of the bundle gets a small stand-in from the core (packages/solara-widget-manager/src/jquery.ts, through a webpack alias):
  • Until the jquery chunk runs, $(node) wraps the DOM node in an array-like object ($el[0], $el.length).
    That is all that @jupyter-widgets/base and Vue views use.
  • Any other use of jQuery throws an error that names the flag.
    This covers jQuery methods on $el (for example view.$el.empty()), $("<div>"), $(function), $.ajax and $._data:
    solara: this widget uses jQuery (.empty()), which this page does not load. Add +jquery to --frontend (SOLARA_FRONTEND), for example --frontend=minimal,+jquery.
  • The browser console shows this error once, and ipywidgets 8 shows it in the error view of the widget.
    The server logs one warning: A widget of the page uses the frontend feature 'jquery', which this server does not load, so the widget fails. Add "+jquery" to --frontend (SOLARA_FRONTEND) to load it, for example --frontend=minimal,+jquery.
  • When the jquery chunk runs, $ forwards everything to the real jQuery.
    This includes plugins that add to $: jQuery UI sets $.ui and $.cleanData, and these land on the real jQuery.
    Backbone.$ (also window.Backbone.$) becomes the real jQuery, as before.
  • full and +jquery run the jquery chunk before the first view, so every view.$el is a real jQuery object, as in feat: load pages faster by splitting the frontend into optional features #1235.
  • When jquery is on but its preloaded chunk did not run (for example after a network error), the widget manager loads the chunk again.
    It does this once, before it loads any widget class.
• The stand-in has no create_view hook, and does not guess which widget needs jQuery.
  It does not change the prototype of $el objects that exist before jQuery loads.
  The earlier version of perf: let minimal pages without an Output widget skip the Jupyter output renderers #1236 (commit b17a9bcb) had all three, and Q45 dropped them.
• The docs table of frontend features has a jquery row.
  The docs say that anywidget widgets (also solara.FigurePlotly with plotly 6 or later), pythreejs and other widgets that use view.$el need +jquery in minimal.
• The benchmark summary knows the jquery feature.

Validation

I built all 8 bundles: Vue 2 and 3, ipywidgets 7 and 8, production and development.
I compared them with the build of the lumino PR (#1240) at ed204719.
I tested on three environments: Vue 3 with ipywidgets 8, Vue 2 with ipywidgets 8, and Vue 2 with ipywidgets 7.
The table shows the build after the review fixes.
Sizes are bytes after gzip -9 -n.
"Full JS" is the core plus every chunk that the full preset preloads.

build	core gzip, lumino PR	core gzip, this PR	saved	jquery gzip	full JS gzip, lumino PR to this PR
Vue 3, ipywidgets 8, prod	170,977	141,897	29,080	30,544	608,315 to 609,780
Vue 3, ipywidgets 7, prod	165,731	136,646	29,085	30,497	594,451 to 595,860
Vue 2, ipywidgets 8, prod	289,266	260,144	29,122	30,543	530,899 to 532,320
Vue 2, ipywidgets 7, prod	283,934	254,861	29,073	30,496	532,135 to 533,555
Vue 3, ipywidgets 8, dev	473,775	392,867	80,908	84,455	1,373,120 to 1,376,671
Vue 2, ipywidgets 8, dev	666,010	585,280	80,730	84,454	1,203,037 to 1,206,764

• In production builds, a full page loads about 1.4 KB gzip more JS than with the lumino PR (perf: let minimal pages skip most of Lumino until a widget needs it #1240), in one more request.
  In development builds, it loads 3.6 to 3.9 KB gzip more.
• The jupyter-controls and output-widget chunks change by less than 10 bytes gzip.
• The jquery chunk holds two modules: jQuery 3.7.1 and the chunk root, which hands jQuery to the stand-in.
• I ran tests/unit/frontend*_test.py (three files) and tests/benchmark/summary_test.py once on each environment: 111 passed on each (104 and 7).
  After the review fixes, I rebuilt and ran the same four files again once on each environment: 111 passed on each.
  The new and changed unit tests are:
  • test_jquery_out_of_core (new) checks all 8 builds.
    The core has no jQuery and has the stand-in, and the jquery chunk has jQuery.
    The lumino, jupyter-controls and output-widget chunks have no jQuery.
  • test_parse_closure (changed) now also checks that +jupyter-controls and +output-widget bring jquery, and that full,-jquery is an error.
  • test_missing_message_logs_once (new) checks that the server logs one warning that names +jquery, and test_missing_message_not_in_full (new) checks that a full server logs nothing.
• I ran 14 browser tests of tests/integration/frontend_chunks_test.py once on each environment, with the flask and the starlette server: 28 passed on each.
  After the review fixes, I ran the same 14 tests plus the new test_full_jquery_request_fails once on each environment: 30 passed on each.
  The new and changed tests are:
  • test_jquery[full] and test_jquery[minimal,+jquery]: a page widget whose view is not Vue based calls view.$el.empty().append(...), .find() and .css(), and works.
    The $el of every DOM view of the page is a real jQuery object, for the Vue view and the other view.
    window.Backbone.$ is jQuery.
    The page requests the jquery chunk once, and shows no warning and no error.
  • test_jquery[minimal]: the same widget fails, and the page requests no jquery chunk.
    The console shows the error that names +jquery once.
    On ipywidgets 8, the error view of the widget shows the same text.
    The server logs one warning that names +jquery.
    A call to Backbone.$._data(...) throws the same kind of error, for $._data.
  • test_full_jquery_request_fails[flask|starlette]: in full, the first request for the jquery chunk fails with a 404.
    The page requests the chunk a second time, the jQuery widget works, and every $el is a real jQuery object.
    The server logs no warning.
    Before the fix, I reproduced the bug in the browser: after the 404, the widget showed the error that tells you to add +jquery. I kept no log of that run.
  • test_minimal_lazy_controls: in minimal, an IntSlider loads jupyter-controls with one warning that names jupyter-controls.
    The jquery chunk loads once with it, without a warning.
    The right arrow key moves the slider from 3 to 4. On ipywidgets 7, this slider is jQuery UI's.
  • test_tab[full] and test_tab[minimal] also check that the page requests jquery once.
  • test_lumino_requirejs has two page widgets. The first one asks requirejs only for modules that the core defines.
    It no longer uses jQuery, so in minimal it still works without jquery, and the page loads no lumino chunk for it.
• The other browser tests were test_full_preloads_sync, test_full_lumino_request_fails, test_full_display_outputs, test_minimal_lazy_output, test_full_output_widget_request_fails and test_minimal_soft_remount_keeps_one_widget_renderer.
• The pre-commit hooks (ruff, mypy, codespell) passed.

Gaps

• I did not measure load or run times on these branches.
  The 3 ms and 15 ms come from an earlier CPU profile of the earlier perf: let minimal pages without an Output widget skip the Jupyter output renderers #1236 builds: 3.2 ms at 1x, and 15.5 to 16 ms at 4x, on Vue 2 and Vue 3.
  That profile ran under Playwright on 10 to 20 cold loads from localhost, so these numbers are estimates of what real users pay.
• Playwright and DevTools overstate the start cost of jQuery.
  jQuery throws and catches one error when it starts, and while DevTools or Playwright is attached, V8 then parses the functions on the stack again.
  In the earlier profile, that cost 46 ms at 1x and 211 ms at 4x on the Vue 3 build of feat: load pages faster by splitting the frontend into optional features #1235.
  I conclude from V8's source that real users do not pay this cost, but I could not measure it without DevTools.
  So a Playwright benchmark such as tests/benchmark will likely show a larger win in minimal than real users get.
  This is an estimate: I did not run the benchmark.
• ipywidgets 7 has no error view.
  There, a widget that uses jQuery in minimal does not show, and only the browser console shows the error.
• The server logs the warning once per server process, as for the other frontend features, not once per browser session.
• In minimal, a view that exists before the controls or the Output widget load jQuery keeps its stand-in $el.
  A widget that uses jQuery and renders before the first control fails, and its error view stays.
  Widgets that render after the first control work.
  A later jQuery call on the old view throws an error that says the page loaded jQuery only after the view was made.
• The stand-in $ is not the same object as jQuery; only Backbone.$ is.
  Code that checks $ === jQuery sees a difference.
  I did not look for such code in other widget libraries.
• When the second load of a failed jquery chunk also fails, the widgets keep the stand-in.
  Their error then says to add +jquery, which the page already has.
  While that second load runs, no widget class loads.
  A request that hangs delays every widget, up to the 120 s chunk timeout of webpack.
• A page whose template has no chunk tags (an old custom template) never preloads jquery.
  jQuery runs there only after a control or the Output widget loads it, so widgets that use jQuery before that fail.
  Their error says to add +jquery, but that cannot help, because such a template preloads no features.
  Before this PR, jQuery was in the core, so these widgets worked there.
• I did not test real anywidget, solara.FigurePlotly or pythreejs widgets.
  The browser test uses a page widget that calls the same $el methods.
  The anywidget notebook extension calls view.$el.empty().

Align results

The #1235 description lists the earlier questions of this work, Q0 to Q46.
These three settle this PR:

• Q41, reviewers: Opus agents do the work, and astra and gpt-6.1-sol do the reviews. Reviews run in parallel with builds, and do not rebuild.
• Q45, plain features in a stack: perf: let minimal pages without an Output widget skip the Jupyter output renderers #1236 becomes the bottom of a GitHub stack with plain features. jQuery and Lumino each get their own PR (this jquery PR and the lumino PR (perf: let minimal pages skip most of Lumino until a widget needs it #1240)): on in full (as in feat: load pages faster by splitting the frontend into optional features #1235), off in minimal, opt in with +jquery and +lumino. jupyter-controls and output-widget bring what their code imports, so here they also need jquery. As a driver default, the create_view hook, the prototype swap and the guessing go. For jQuery, this deviates from Q37 (a widget always works) for widgets in minimal that are not Vue based, by Maarten's call. It also reverses the Q39 outcome that jQuery stays in the core.
• Q47, jQuery opt-in and Lumino lazy: jQuery is opt-in (option C): a clear error that names +jquery, and no lazy load. Maarten accepted that anywidget, solara.FigurePlotly and pythreejs need +jquery in minimal. Lumino loads lazily with one warning that names the module that asked for it, in the lumino PR (perf: let minimal pages skip most of Lumino until a widget needs it #1240). The semver port is dropped. As a driver default, the stand-ins for @jupyterlab/coreutils, the kernelspec API and minimist are dropped too. The work file gives "under 2 KB, under 1 ms" as the reason. An earlier measurement put the minimist and kernelspec stand-ins at about 1.6 KB gzip and under 0.5 ms at normal CPU speed. Without the stand-ins, all of @jupyterlab/coreutils and @lumino/polling stay in the core. Nobody measured that total on these branches.

Crossreview results

Two rounds, with two reviewers: astra and gpt-6.1-sol.
Both come from one vendor, OpenAI. Maarten chose them in Q41.

Round 1: both reviewed the one commit of the first version (5995f5de) from the commit alone, without builds or tests.
astra found one MEDIUM finding, and gpt-6.1-sol found one MEDIUM and one LOW finding.
Both defects were real.
I reproduced each of them in the browser before I fixed it, but kept no logs of these runs.

• MEDIUM, gpt-6.1-sol: wait for an enabled jquery chunk before views are made.
  When the preloaded jquery chunk failed once in full or minimal,+jquery, the page kept the stand-in for good.
  A widget that uses view.$el then failed with an error that told the user to add +jquery, which the page already had.
  • Fix: commit 2342d5b1. The widget manager loads the chunk again, once, before it loads any model or view class. The new test test_full_jquery_request_fails covers it.
  • Why it got in: the stack tests a failed preload for lumino and output-widget. Those features load on first use, so they recover on their own. jquery is the first feature that never loads on first use, and the jquery commit did not add the same failure test for it.
  • Constitution change: add the same tests for a new member of a family that has a test per member, or say in the PR why not. For example, each frontend feature with a preloaded chunk has a test where the request for that chunk fails.
• MEDIUM, astra, and LOW, gpt-6.1-sol (one defect): only known names should skip the +jquery error.
  Without jquery, the stand-in let every name that starts with _ pass as a probe.
  A call such as Backbone.$._data(node, "events") then failed with is not a function, without the +jquery error, and the server logged nothing.
  • Fix: commit fdd9afd3. The four functions of jQuery 3.7 that start with _ (_data, _removeData, _queueHooks, _evalUrl) now count as a use of jQuery. The test_jquery[minimal] test checks the error for $._data.
  • I did not use the allowlist of probe names that both reviewers proposed. With an allowlist, a probe name that the list does not know (from Vue or another library) throws, and that breaks Vue widgets in minimal.
  • Why it got in: the stand-in's pattern was written from the side of the code that probes objects, and nobody checked it against the names that jQuery itself has.
  • Constitution change: when a stand-in lets names through by a pattern, check the pattern against the full API of the library that it replaces.
    Test each name that both match.

Fix sizes:

• fdd9afd3 changes 8 lines (4 in jquery.ts, 4 in the test).
• 2342d5b1 changes 66 lines (22 in jquery.ts, 6 in manager.ts, 38 in the test), and adds the exported function enabledJQueryLoaded.

Round 2: astra and gpt-6.1-sol reviewed 2342d5b1 (the retry of the jquery chunk, 66 lines) again, and found nothing.
They reviewed that commit alone, not the whole diff of this PR.
astra, which raised the finding, confirmed fdd9afd3 (8 lines: $._data and the other three functions now name +jquery) as a small fix, with no new defect.
astra and an Opus agent also checked the PR descriptions against the code, and this version of the description has their corrections.

🤖 Generated with Claude Code