You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Context as of July 2026 — read this first. The document below was written as a review of the Bookshelf codebase as deployed, but the situation has moved on since the events it anticipates: the Bookshelf/Notes EC2 server was switched off on 11 Feb 2025 (see #145, with the AWS teardown and backups recorded in #151), so the direct hosting saving (~USD $50/month) is already banked. notes.quantecon.org currently redirects to the static notebook-gallery repo, which holds the archived .ipynb files behind a plain README — no rendered pages, browse/filter, search, or comments.
That changes the pitch of this proposal from "cut costs" to "restore the functionality we lost, at ~zero running cost" — it is effectively the follow-up that #145 closed with ("We will look at opportunities in the new mystmd space to see if we can support a more cost effective notes.quantecon.org system"). It also simplifies the migration plan in §9: the primary content source can be the notebooks already sitting in notebook-gallery/ipynb, with the final mongodump as the fallback for metadata, author identities, and archived comments.
Prepared July 2026 · Based on a code review of QuantEcon/Bookshelf at commit 2fb135e (June 2019)
1. Executive summary
Bookshelf is a classic three-tier dynamic web application: a React single-page app, an Express/Node API, and a MongoDB database, all running as Docker containers on a single EC2 medium instance. Every feature — publishing, rendering, search, comments, votes, views, notifications, moderation — flows through that one always-on server and database.
That coupling is the root cost driver. The instance must be sized for MongoDB's memory appetite plus notebook rendering, must run 24/7 even though content changes rarely, and every subsystem (OAuth apps, sessions, uploads, email, backups, OS, and ~75 npm dependencies frozen in 2019) is a maintenance obligation for a small team.
The good news: every one of Bookshelf's concerns now has a free, managed, git-native counterpart. The proposed v2 treats a GitHub repository as the content database, GitHub Actions as the rendering pipeline, a static host (GitHub Pages or Cloudflare Pages) as the server, GitHub Discussions (via giscus) as the comment and voting system, and a build-time search index (Pagefind) as the search engine. Estimated running cost drops from roughly $400–550/yr plus significant maintenance hours to approximately the price of the domain name (~$12/yr) with near-zero steady-state operations.
The honest tradeoffs: commenting requires a GitHub account, vote-based sorting refreshes on a schedule rather than live, per-user email preferences become GitHub's native subscription model, and publishing takes a CI run (~2–5 minutes) instead of being instant. For QuantEcon's audience — computational economists who almost universally have GitHub accounts — these are mild costs for a dramatic reduction in burden.
Uploaded .ipynb files (/files) and pre-rendered HTML live on the EC2 volume.
Email
Mailgun
Comment/reply/submission notifications with per-user settings.
Deployment
docker-compose (2 containers) on one EC2 medium; client build/ volume-mounted into the server container
Daily backups via backup.sh: mongodump → tarball → committed to a git repo.
2.2 Data model (from server/js/db/models/)
The Submission document is the heart of the system, and it contains a striking design decision: the entire notebook is stored inside the MongoDB document as a JSON string (notebookJSONString), alongside a path to the original .ipynb on local disk. Metadata includes title, summary, a fixed topic taxonomy (~24 economics fields, JEL-style), language, author and co-authors, plus engagement state: score (votes), views, a viewers array of user ObjectIds for unique-viewer counting, comments (ObjectId list), totalComments, and moderation flags (flagged, flaggedReason, deleted).
Comment documents support one level of threading (replies, isReply, parentID), editing, deletion, votes, and flagging. User documents store multi-provider identity — including, notably, OAuth access_tokens for each linked provider stored in plaintext in the database — along with avatar handling, bio/links, email notification settings, and vote history (upvotes/downvotes arrays).
2.3 Feature inventory (from the routes)
Reading server/app.js and server/routes/ gives the definitive feature list. This is the checklist any rebuild has to consciously keep, adapt, or drop:
Browse & discover — listing with filters (language, topic), five sort orders (Votes, Comments, Viewers, Discover, Date), pagination.
Full-text search — MongoDB text index weighted title:2, authorName:2, summary:1.
Votes — up/downvote on submissions and comments; drives the Votes sort.
Views — total views plus unique authenticated viewers.
Auth — GitHub, Google, Twitter OAuth; JWT for API calls; session management; account restore.
Notifications — Mailgun emails on new comment/reply/submission, per-user opt-outs.
Moderation & admin — flagging with reasons, admin allow-list (capped at 5), site-wide announcements, an invite system.
SEO — generated sitemap.xml, templated robots.txt, meta tags, and an optional preRender mode (nbconvert → HTML on disk) because an SPA is otherwise crawler-hostile.
2.4 How a page view actually works (and why it matters)
The /api/search/notebook/:nbid handler is instructive. On every read it: fetches the submission including the full notebookJSONString, increments views, appends to and dedupes the viewers array, recomputes totalComments by loading all comment documents and counting replies, and saves the submission back to MongoDB — then ships the entire notebook JSON to the browser for nteract + MathJax to render client-side.
So the hot path does multiple database writes per page view and serves multi-megabyte JSON payloads for what is, semantically, a static document. This is the single clearest illustration of why the system needed a medium instance (working memory for Mongo plus fat request handling) and why page loads were slower than a static site would be. v1 pays rendering and database costs on every view; a static architecture pays rendering once per edit, at build time.
3. Why it was expensive: cost & maintenance analysis
3.1 Direct infrastructure cost
An always-on t3.medium (2 vCPU / 4 GB) runs roughly $30/month on-demand, before EBS storage (notebooks, rendered HTML, and the Mongo data directory all live on the instance volume), snapshots, and data transfer. All-in, a realistic estimate is $35–50/month ≈ $420–600/year. (Figures approximate; worth re-checking current AWS pricing.) The instance can't be downsized much: MongoDB wants RAM for its working set, and the 50 MB upload limit plus whole-notebook JSON handling means request spikes are memory-hungry. A micro would OOM; you were paying for headroom that sat idle almost all the time, because content changes are rare but the server model forces 24/7 provisioning for peak.
3.2 The bigger cost: operations for a small team
The direct AWS bill was likely the smaller half of the true cost. The architecture creates a long list of standing obligations:
A stateful snowflake server. Database, uploaded files, and rendered HTML all live on one instance's disk. Upgrades, migrations, and disaster recovery are all manual and risky. The backup strategy — nightly mongodump tarballs committed to a git repository — is a symptom: it works, but someone has to own it, verify restores, and watch that repo grow unboundedly.
Security surface. Three live OAuth applications (each with credentials, callback URLs, and provider policy churn — Twitter's API upheaval alone would have broken that login path), session secrets, JWTs, plaintext provider access tokens in the database, a 50 MB unauthenticated-adjacent upload path, and Mailgun keys.
Dependency rot. The stack froze in June 2019: Node 9 (end-of-life the same month), Passport 0.4.x (session-fixation vulnerability CVE-2022-25896, fixed only in 0.6), mongoose 5, CRA 2, MathJax 2. ~75 runtime dependencies across client and server means a constant stream of CVE notifications, and today the app effectively cannot be patched — it would need a rewrite just to get back to a maintainable baseline. That makes this the ideal moment to change the architecture rather than refresh it.
One risky default worth flagging:docker-compose.yml publishes MongoDB as 27017:27017, binding it on the host. Any security-group slip would have exposed an unauthenticated database to the internet. Nothing suggests it happened — but it's the kind of latent risk a small team shouldn't have to carry at all.
Team-shaped fragility. 784 commits, 2017–2019, mostly from student contributors who have since moved on. Institutional knowledge of the deploy dance (stop-docker → pull → rebuild client → rebuild image → restart) leaves with the people.
3.3 The structural lesson
Bookshelf bundled four separable concerns — content storage, rendering, discussion, and discovery — into one server and one database. Each concern individually now has a free managed service that a small team never has to patch. The rebuild below is essentially the act of un-bundling them.
4. Design goals for v2
Derived from the review and your constraints: (1) no always-on servers or databases to run; (2) hosting and tooling within free tiers, so steady-state cost ≈ domain renewal; (3) keep the essence — a browsable, searchable, commentable gallery of community notebooks; (4) align with QuantEcon's existing muscle memory: static site generators, GitHub-centric workflows, CI-driven publishing; (5) content and discussion must remain exportable (no data hostage-taking); (6) equal or better page speed and SEO than v1.
5. Recommended architecture (Option A): the git-native static site
One sentence: a public GitHub repo is the database, GitHub Actions is the render farm, a static host is the server, GitHub Discussions is the comment/vote system, and a build-time index is the search engine.
One directory per note keeps everything self-describing and trivially exportable:
notes/
smith-optimal-savings/
notebook.ipynb
meta.yml
site/ # theme, templates, landing pages
data/
engagement.json # written by the nightly reactions sync (see 5.5)
redirects.yml # legacy Mongo-id → slug map (see §9)
meta.yml replaces the Mongo submission document's metadata fields:
title: "Optimal Savings with JAX"authors:
- name: Jane Smithgithub: jsmith # optional; enables avatar + profile linksummary: > Solves the household savings problem with EGM on GPU.topics: [Macroeconomics, Computational Economics] # same fixed taxonomy as v1language: python # was `lang` in v1date: 2026-07-24license: BSD-3-Clause # explicit, which v1 never captured
The three engagement fields that lived in Mongo (score, views, comments) move to external systems (§5.4–5.6); the repo stays purely content. Git history replaces lastUpdated, published, deleted, and the entire backup apparatus — the daily mongodump-to-git ritual becomes a no-op, because the data already lives in git.
Notebook size policy: enforce a cap in CI (e.g. 10–25 MB with outputs included). Committing outputs keeps the pipeline simple and rendering instant; stripping outputs and re-executing in CI is a possible later upgrade (better reproducibility, but it imports the whole "does this run, with which package versions, in how many CI minutes" problem — QuantEcon knows that world well from the lectures, and it shouldn't block v2.0).
5.2 Submission flows: three tiers of friction
Tier 1 — Pull request (primary). Contributor adds a directory, opens a PR. CI validates (nbformat well-formed, size cap, meta.yml schema, license present, kernelspec sane) and posts a preview deployment (automatic on Cloudflare Pages; achievable on GH Pages via artifact links). Maintainer review is the moderation gate — strictly better than v1's flag-after-publish model. Merge = published.
Tier 2 — GitHub Issue Form (low friction). A Submit a notebook issue template with structured fields: title, summary, topic dropdown (the taxonomy above), language, and a link to the notebook (gist, repo, Colab) or a .zip attachment — note GitHub issues won't accept a bare .ipynb attachment, but they do accept .zip. A label-triggered Action (or a maintainer with a helper script) turns the issue into a PR. This serves the "I have a notebook and an email address" audience the old upload form served, minus the anonymous-upload attack surface.
Tier 3 — Web upload form (optional, later). If issue-form friction proves real: a small Cloudflare Pages Function (~100 lines) that accepts an upload, validates it, and opens a PR via a GitHub App token. Still $0 on the free tier (100k requests/day). I'd consciously defer this — every dynamic sprinkle re-imports a little of the old ops burden — but it's the escape hatch if Tiers 1–2 don't cover everyone.
5.3 Rendering: three credible engines
All three run identically well in Actions; this is the main technology choice to spike before committing.
QuantEcon is a core driver of this ecosystem; shared theme with lecture sites
Minimal deps, maximal control, most code to own
Recommendation: prototype both Quarto and MyST on ~10 real migrated notebooks. Quarto is the fastest path to a finished gallery (listings + search + giscus are configuration, not code). MyST maximizes ecosystem alignment and long-term theme sharing with the lecture sites, at the cost of writing a modest listing/index generator (a ~100-line Python script over the meta.yml files that emits the landing page — well within the team's comfort zone). Both beat hand-rolled nbconvert unless you want total control of every pixel.
5.4 Comments: giscus on GitHub Discussions
giscus embeds a GitHub Discussions thread on each page: one Discussion per note (auto-created on first comment, mapped by page path), threaded replies, markdown with math, reactions, edit/delete — feature-parity with v1's comment system, minus down-votes. Sign-in is GitHub OAuth handled entirely by giscus; the site itself holds zero credentials, sessions, or user data. Moderation uses GitHub's native tools (lock, delete, block, report), replacing the flag/admin routes. Notifications become GitHub's subscribe/watch semantics — anyone who comments is auto-subscribed and gets email through their own GitHub preferences, which retires Mailgun, the templates, and per-user settings storage in one stroke.
Alternatives considered: utterances (Issues-based, flat threads — fine but Discussions is the better-shaped home), and Discourse embedding — worth naming because if QuantEcon still operates a Discourse forum anyway, embedding forum topics as comment threads is a well-trodden pattern that adds community cross-traffic; but standing up Discourse just for this would recreate the server problem. Hypothes.is could optionally be layered on for inline annotation of notebook content — a genuinely nice fit for scholarly notes and also $0/free-tier.
5.5 Votes and sorting: reactions, synced at build time
The 👍 reaction on each note's Discussion is the vote. A scheduled Action (nightly, or on-demand) queries the GitHub GraphQL API for reaction and comment counts across the Discussions category, writes data/engagement.json, and triggers a rebuild — so listing pages can offer sort by votes / comments / date, matching three of v1's five sort orders. Viewers sorting depends on view counts (below); the Discover shuffle is trivially reproducible at build time. The tradeoff is freshness: rankings update daily, not instantly. For a notes gallery, that's almost certainly acceptable — and it's tunable (run the sync hourly if it ever matters; it's one cron line).
5.6 Views: outsource or drop
Options in ascending order of effort: (a) drop view counts entirely; (b) Cloudflare Web Analytics or GoatCounter — both free, privacy-friendly, no cookie banner; GoatCounter has an API you can query in the nightly sync to bake per-page view counts into listings, restoring the Viewers sort if wanted. Unique-authenticated-viewer counting (v1's viewers array) has no static equivalent and should be consciously dropped — it was also the feature causing database writes on every read.
5.7 Search: Pagefind
Pagefind indexes the rendered HTML at the end of the build and serves search from static files with a tiny (~100 kB-scale) runtime payload, scaling comfortably to thousands of pages. Field weighting (boost title/author over body) reproduces v1's weighted Mongo text index; filters map to topic/language facets. If Quarto is chosen, its built-in search may suffice and Pagefind becomes optional.
5.8 Hosting
Either GitHub Pages (maximum simplicity, consistent with other QuantEcon sites) or Cloudflare Pages (faster global CDN, preview deployments per PR — genuinely useful for reviewing submissions, and server-side _redirects support, which matters for migration §9). Both: $0, automatic TLS, custom domain. Mild preference for Cloudflare Pages purely for PR previews + redirects; GH Pages is a fine default if you'd rather stay single-vendor.
Actions is free for public repos; this entire pipeline is well inside free-tier minutes even rebuilding nightly.
6. Option B: static core + serverless sprinkle
Option A plus the Tier-3 upload Worker, and optionally a Workers KV/D1 "like" counter that doesn't require GitHub login. Still ~$0 within Cloudflare's free tier. My advice is to hold this in reserve: each dynamic endpoint reintroduces secrets, abuse handling, and a thing that can be down. Adopt only if submission friction or "votes require GitHub" produces real, observed complaints.
7. Option C: stay dynamic but cheaper (assessed and not recommended)
For completeness: the v1 feature set could be rehosted on Fly.io/Railway/Render (~$5–15/mo) with MongoDB Atlas M0 (free) or SQLite + Litestream. This cuts the AWS bill by ~70% but keeps ~100% of the real cost — the dependency treadmill, the OAuth apps, the security surface, the backup ownership. Given the 2019 codebase needs a rewrite to be maintainable anyway, spending that rewrite on a server app rebuilds the problem. Not recommended.
8. Feature-by-feature mapping: v1 → v2
v1 feature
v1 mechanism
v2 mechanism
What changes
Publish a notebook
Upload form → Express → Mongo + disk
PR or issue form → merge → CI render
Publish latency ~2–5 min; review happens before publish
Notebook page
Client-side nteract render of JSON from API
Pre-rendered static HTML
Faster loads, real SEO; no 50 MB payloads
Browse: filter by topic/language
Mongo queries
Listing pages + client-side facet filter
Equivalent
Sort by votes/comments/date
Mongo sort on live counters
Build-time sort from nightly engagement.json
Day-level freshness instead of live
Sort by viewers
viewers array in Mongo
GoatCounter API in nightly sync, or drop
Approximate, or consciously dropped
Full-text search
Mongo text index (weighted)
Pagefind / Quarto search (weighted)
Equivalent; runs client-side
Comments + replies + math
Custom Mongo comments, react-markdown
giscus ⇄ GitHub Discussions
Requires GitHub login; better moderation tools
Up/downvotes
Custom vote routes
👍 reactions on Discussions
Upvotes only; no downvotes
Email notifications
Mailgun + per-user settings
GitHub watch/subscribe
Users manage prefs on GitHub; zero email infra
Auth (GitHub/Google/Twitter)
Passport, JWT, sessions, stored tokens
None on the site; giscus handles GitHub OAuth
Google/Twitter login gone; no credentials held
Moderation
Flag routes, admin list, delete/restore
PR review gate + Discussions moderation
Strictly stronger (pre-publish review)
Announcements
Admin CRUD in Mongo
A markdown banner include in the repo
Edit a file
Download .ipynb
API endpoint
Static file link next to the page
Simpler
Sitemap/SEO
Generated sitemap, SPA prerender flag
Native static HTML + SSG sitemap
Strictly better
Backups
Nightly mongodump → git repo
The repo is the data; Discussions exportable via API
Obligation disappears
Consciously dropped: unique-authenticated-viewer counts, downvotes, the invite system (obsoleted by PR review), per-user profile pages with bios (author identity becomes the GitHub profile linked from meta.yml; a per-author listing page is easy build-time glue if wanted).
9. Migration plan
Export. One-off script against the final mongodump: for each non-deleted submission, write notebook.ipynb from notebookJSONString (with a fallback to the archived .ipynb files from the instance's /files volume) and meta.yml from the metadata fields. The User collection's github.username field lets you attach real GitHub identities to many authors automatically.
Validate. CI renders every migrated notebook; triage the failures (there will be a few — old nbformat versions, exotic outputs). This doubles as the renderer bake-off corpus from §5.3.
Preserve URLs. Build a redirects.yml mapping old /submission/<mongo-id> paths to new slugs. On Cloudflare Pages this becomes a _redirects file (real 301s); on GitHub Pages, generate meta-refresh + rel=canonical stub pages. Inbound links and search ranking survive.
Archive old discussions. Render each submission's existing comments (from the dump) as a static "Archived comments (2017–2019)" section on the page, and let giscus start fresh below it. This is more honest than replaying comments into Discussions under a bot account, and it's zero-risk. (Replaying via the GraphQL API is possible if continuity is preferred — but all migrated comments would show as bot-authored with attribution lines.)
Cut over. Freeze the old site read-only, point DNS at the static host, keep the final mongodump archived. Decommission the EC2 instance — that's the moment the meter stops.
(AWS figures are approximate on-demand estimates; the ops row is where the real savings live.)
11. Risks and open questions
GitHub login as the price of participation. Commenting and voting require GitHub accounts. For this community that's nearly universal — and v1 already required a social login — but it's the biggest philosophical change. The Discourse-embed variant (§5.4) is the fallback if broader identity support ever matters.
Comment data lives in GitHub. Exportable via API at any time (worth adding a quarterly export to the nightly workflow for belt-and-braces), but it's a dependency on GitHub's product decisions. giscus itself is MIT-licensed and self-hostable if its hosted instance ever changes.
Repo growth from notebook outputs. Mitigated by the size cap; output-stripping + CI execution is the long-term fix if it bites. Avoid Git LFS on GitHub Pages paths (bandwidth quotas) — prefer smaller notebooks.
Build-time engagement sorting means a vote cast today reorders listings tomorrow. Tunable via cron frequency; decide what "fresh enough" means.
Spam/abuse shifts to GitHub's layer (Discussions moderation, PR review). Materially better than v1's post-hoc flagging, but maintainers should enable Discussions moderation settings from day one.
Tooling drift. The static-site tool space (mystmd especially) moves quickly; re-verify current capabilities during the spike rather than trusting this document's snapshot.
12. Suggested next steps
Week 1–2 — renderer spike. Migrate ~10 representative notebooks with the export script; build the same mini-gallery twice (Quarto vs MyST). Decide on renderer + host.
Week 2–3 — full migration dry run. Run the exporter over the complete dump; fix render failures; generate the redirect map.
Cutover. DNS switch, old site read-only for a grace period, then terminate the instance.
Later, only if pulled by demand: upload Worker (Tier 3), hourly engagement sync, CI re-execution of notebooks, per-author pages, Hypothes.is annotations.
The strategic shape of this rebuild: v1 was a product your team had to operate; v2 is a pipeline your team curates. Everything that used to page someone at 2 a.m. — the database, the sessions, the uploads, the email, the backups — either becomes a file in git or someone else's managed service, and the notes themselves end up faster to load, easier to find, and reviewed before they're published.
Note
Context as of July 2026 — read this first. The document below was written as a review of the Bookshelf codebase as deployed, but the situation has moved on since the events it anticipates: the Bookshelf/Notes EC2 server was switched off on 11 Feb 2025 (see #145, with the AWS teardown and backups recorded in #151), so the direct hosting saving (~USD $50/month) is already banked. notes.quantecon.org currently redirects to the static notebook-gallery repo, which holds the archived
.ipynbfiles behind a plain README — no rendered pages, browse/filter, search, or comments.That changes the pitch of this proposal from "cut costs" to "restore the functionality we lost, at ~zero running cost" — it is effectively the follow-up that #145 closed with ("We will look at opportunities in the new
mystmdspace to see if we can support a more cost effective notes.quantecon.org system"). It also simplifies the migration plan in §9: the primary content source can be the notebooks already sitting innotebook-gallery/ipynb, with the final mongodump as the fallback for metadata, author identities, and archived comments.QuantEcon Notes (Bookshelf): Architecture Review & Low-Cost Rebuild Proposal
Prepared July 2026 · Based on a code review of QuantEcon/Bookshelf at commit
2fb135e(June 2019)1. Executive summary
Bookshelf is a classic three-tier dynamic web application: a React single-page app, an Express/Node API, and a MongoDB database, all running as Docker containers on a single EC2
mediuminstance. Every feature — publishing, rendering, search, comments, votes, views, notifications, moderation — flows through that one always-on server and database.That coupling is the root cost driver. The instance must be sized for MongoDB's memory appetite plus notebook rendering, must run 24/7 even though content changes rarely, and every subsystem (OAuth apps, sessions, uploads, email, backups, OS, and ~75 npm dependencies frozen in 2019) is a maintenance obligation for a small team.
The good news: every one of Bookshelf's concerns now has a free, managed, git-native counterpart. The proposed v2 treats a GitHub repository as the content database, GitHub Actions as the rendering pipeline, a static host (GitHub Pages or Cloudflare Pages) as the server, GitHub Discussions (via giscus) as the comment and voting system, and a build-time search index (Pagefind) as the search engine. Estimated running cost drops from roughly $400–550/yr plus significant maintenance hours to approximately the price of the domain name (~$12/yr) with near-zero steady-state operations.
The honest tradeoffs: commenting requires a GitHub account, vote-based sorting refreshes on a schedule rather than live, per-user email preferences become GitHub's native subscription model, and publishing takes a CI run (~2–5 minutes) instead of being instant. For QuantEcon's audience — computational economists who almost universally have GitHub accounts — these are mild costs for a dramatic reduction in burden.
2. What Bookshelf is: system overview
2.1 Components
@nteract/notebook-preview, MathJax 2, react-bootstrap, SCSS via gulpDockerfile-server), Express 4, Passport (GitHub, Google, Twitter strategies; Facebook present but disabled), express-session + JWT, multer/multiparty/connect-multiparty uploads, mailgun-js, mongoose 5, sitemapjupyter nbconvertfor optional server-side pre-rendering.mongo:latestimage)users,submissions,comments,adminlists,announcements,emaillists..ipynbfiles (/files) and pre-rendered HTML live on the EC2 volume.medium; clientbuild/volume-mounted into the server containerbackup.sh:mongodump→ tarball → committed to a git repo.2.2 Data model (from
server/js/db/models/)The
Submissiondocument is the heart of the system, and it contains a striking design decision: the entire notebook is stored inside the MongoDB document as a JSON string (notebookJSONString), alongside a path to the original.ipynbon local disk. Metadata includes title, summary, a fixed topic taxonomy (~24 economics fields, JEL-style), language, author and co-authors, plus engagement state:score(votes),views, aviewersarray of user ObjectIds for unique-viewer counting,comments(ObjectId list),totalComments, and moderation flags (flagged,flaggedReason,deleted).Commentdocuments support one level of threading (replies,isReply,parentID), editing, deletion, votes, and flagging.Userdocuments store multi-provider identity — including, notably, OAuthaccess_tokens for each linked provider stored in plaintext in the database — along with avatar handling, bio/links, email notification settings, and vote history (upvotes/downvotesarrays).2.3 Feature inventory (from the routes)
Reading
server/app.jsandserver/routes/gives the definitive feature list. This is the checklist any rebuild has to consciously keep, adapt, or drop:Votes,Comments,Viewers,Discover,Date), pagination.title:2, authorName:2, summary:1..ipynbdownload (/notebook_json/:nbid)..ipynb(body-parser limit set to 50 MB) with a two-step submit/confirm flow, plus edit and delete.Votessort.sitemap.xml, templatedrobots.txt, meta tags, and an optionalpreRendermode (nbconvert → HTML on disk) because an SPA is otherwise crawler-hostile.2.4 How a page view actually works (and why it matters)
The
/api/search/notebook/:nbidhandler is instructive. On every read it: fetches the submission including the fullnotebookJSONString, incrementsviews, appends to and dedupes theviewersarray, recomputestotalCommentsby loading all comment documents and counting replies, and saves the submission back to MongoDB — then ships the entire notebook JSON to the browser for nteract + MathJax to render client-side.So the hot path does multiple database writes per page view and serves multi-megabyte JSON payloads for what is, semantically, a static document. This is the single clearest illustration of why the system needed a
mediuminstance (working memory for Mongo plus fat request handling) and why page loads were slower than a static site would be. v1 pays rendering and database costs on every view; a static architecture pays rendering once per edit, at build time.3. Why it was expensive: cost & maintenance analysis
3.1 Direct infrastructure cost
An always-on
t3.medium(2 vCPU / 4 GB) runs roughly $30/month on-demand, before EBS storage (notebooks, rendered HTML, and the Mongo data directory all live on the instance volume), snapshots, and data transfer. All-in, a realistic estimate is $35–50/month ≈ $420–600/year. (Figures approximate; worth re-checking current AWS pricing.) The instance can't be downsized much: MongoDB wants RAM for its working set, and the 50 MB upload limit plus whole-notebook JSON handling means request spikes are memory-hungry. Amicrowould OOM; you were paying for headroom that sat idle almost all the time, because content changes are rare but the server model forces 24/7 provisioning for peak.3.2 The bigger cost: operations for a small team
The direct AWS bill was likely the smaller half of the true cost. The architecture creates a long list of standing obligations:
mongodumptarballs committed to a git repository — is a symptom: it works, but someone has to own it, verify restores, and watch that repo grow unboundedly.docker-compose.ymlpublishes MongoDB as27017:27017, binding it on the host. Any security-group slip would have exposed an unauthenticated database to the internet. Nothing suggests it happened — but it's the kind of latent risk a small team shouldn't have to carry at all.stop-docker→ pull → rebuild client → rebuild image → restart) leaves with the people.3.3 The structural lesson
Bookshelf bundled four separable concerns — content storage, rendering, discussion, and discovery — into one server and one database. Each concern individually now has a free managed service that a small team never has to patch. The rebuild below is essentially the act of un-bundling them.
4. Design goals for v2
Derived from the review and your constraints: (1) no always-on servers or databases to run; (2) hosting and tooling within free tiers, so steady-state cost ≈ domain renewal; (3) keep the essence — a browsable, searchable, commentable gallery of community notebooks; (4) align with QuantEcon's existing muscle memory: static site generators, GitHub-centric workflows, CI-driven publishing; (5) content and discussion must remain exportable (no data hostage-taking); (6) equal or better page speed and SEO than v1.
5. Recommended architecture (Option A): the git-native static site
One sentence: a public GitHub repo is the database, GitHub Actions is the render farm, a static host is the server, GitHub Discussions is the comment/vote system, and a build-time index is the search engine.
5.1 Content store: the repository as CMS
One directory per note keeps everything self-describing and trivially exportable:
meta.ymlreplaces the Mongo submission document's metadata fields:The three engagement fields that lived in Mongo (
score,views,comments) move to external systems (§5.4–5.6); the repo stays purely content. Git history replaceslastUpdated,published,deleted, and the entire backup apparatus — the daily mongodump-to-git ritual becomes a no-op, because the data already lives in git.Notebook size policy: enforce a cap in CI (e.g. 10–25 MB with outputs included). Committing outputs keeps the pipeline simple and rendering instant; stripping outputs and re-executing in CI is a possible later upgrade (better reproducibility, but it imports the whole "does this run, with which package versions, in how many CI minutes" problem — QuantEcon knows that world well from the lectures, and it shouldn't block v2.0).
5.2 Submission flows: three tiers of friction
Tier 1 — Pull request (primary). Contributor adds a directory, opens a PR. CI validates (nbformat well-formed, size cap, meta.yml schema, license present, kernelspec sane) and posts a preview deployment (automatic on Cloudflare Pages; achievable on GH Pages via artifact links). Maintainer review is the moderation gate — strictly better than v1's flag-after-publish model. Merge = published.
Tier 2 — GitHub Issue Form (low friction). A
Submit a notebookissue template with structured fields: title, summary, topic dropdown (the taxonomy above), language, and a link to the notebook (gist, repo, Colab) or a.zipattachment — note GitHub issues won't accept a bare.ipynbattachment, but they do accept.zip. A label-triggered Action (or a maintainer with a helper script) turns the issue into a PR. This serves the "I have a notebook and an email address" audience the old upload form served, minus the anonymous-upload attack surface.Tier 3 — Web upload form (optional, later). If issue-form friction proves real: a small Cloudflare Pages Function (~100 lines) that accepts an upload, validates it, and opens a PR via a GitHub App token. Still $0 on the free tier (100k requests/day). I'd consciously defer this — every dynamic sprinkle re-imports a little of the old ops burden — but it's the escape hatch if Tiers 1–2 don't cover everyone.
5.3 Rendering: three credible engines
All three run identically well in Actions; this is the main technology choice to spike before committing.
.ipynbwithout executing<script>via template includeRecommendation: prototype both Quarto and MyST on ~10 real migrated notebooks. Quarto is the fastest path to a finished gallery (listings + search + giscus are configuration, not code). MyST maximizes ecosystem alignment and long-term theme sharing with the lecture sites, at the cost of writing a modest listing/index generator (a ~100-line Python script over the
meta.ymlfiles that emits the landing page — well within the team's comfort zone). Both beat hand-rolled nbconvert unless you want total control of every pixel.5.4 Comments: giscus on GitHub Discussions
giscus embeds a GitHub Discussions thread on each page: one Discussion per note (auto-created on first comment, mapped by page path), threaded replies, markdown with math, reactions, edit/delete — feature-parity with v1's comment system, minus down-votes. Sign-in is GitHub OAuth handled entirely by giscus; the site itself holds zero credentials, sessions, or user data. Moderation uses GitHub's native tools (lock, delete, block, report), replacing the flag/admin routes. Notifications become GitHub's subscribe/watch semantics — anyone who comments is auto-subscribed and gets email through their own GitHub preferences, which retires Mailgun, the templates, and per-user settings storage in one stroke.
Alternatives considered: utterances (Issues-based, flat threads — fine but Discussions is the better-shaped home), and Discourse embedding — worth naming because if QuantEcon still operates a Discourse forum anyway, embedding forum topics as comment threads is a well-trodden pattern that adds community cross-traffic; but standing up Discourse just for this would recreate the server problem. Hypothes.is could optionally be layered on for inline annotation of notebook content — a genuinely nice fit for scholarly notes and also $0/free-tier.
5.5 Votes and sorting: reactions, synced at build time
The 👍 reaction on each note's Discussion is the vote. A scheduled Action (nightly, or on-demand) queries the GitHub GraphQL API for reaction and comment counts across the Discussions category, writes
data/engagement.json, and triggers a rebuild — so listing pages can offer sort by votes / comments / date, matching three of v1's five sort orders.Viewerssorting depends on view counts (below); theDiscovershuffle is trivially reproducible at build time. The tradeoff is freshness: rankings update daily, not instantly. For a notes gallery, that's almost certainly acceptable — and it's tunable (run the sync hourly if it ever matters; it's one cron line).5.6 Views: outsource or drop
Options in ascending order of effort: (a) drop view counts entirely; (b) Cloudflare Web Analytics or GoatCounter — both free, privacy-friendly, no cookie banner; GoatCounter has an API you can query in the nightly sync to bake per-page view counts into listings, restoring the
Viewerssort if wanted. Unique-authenticated-viewer counting (v1'sviewersarray) has no static equivalent and should be consciously dropped — it was also the feature causing database writes on every read.5.7 Search: Pagefind
Pagefind indexes the rendered HTML at the end of the build and serves search from static files with a tiny (~100 kB-scale) runtime payload, scaling comfortably to thousands of pages. Field weighting (boost title/author over body) reproduces v1's weighted Mongo text index; filters map to topic/language facets. If Quarto is chosen, its built-in search may suffice and Pagefind becomes optional.
5.8 Hosting
Either GitHub Pages (maximum simplicity, consistent with other QuantEcon sites) or Cloudflare Pages (faster global CDN, preview deployments per PR — genuinely useful for reviewing submissions, and server-side
_redirectssupport, which matters for migration §9). Both: $0, automatic TLS, custom domain. Mild preference for Cloudflare Pages purely for PR previews + redirects; GH Pages is a fine default if you'd rather stay single-vendor.5.9 The build, concretely
Actions is free for public repos; this entire pipeline is well inside free-tier minutes even rebuilding nightly.
6. Option B: static core + serverless sprinkle
Option A plus the Tier-3 upload Worker, and optionally a Workers KV/D1 "like" counter that doesn't require GitHub login. Still ~$0 within Cloudflare's free tier. My advice is to hold this in reserve: each dynamic endpoint reintroduces secrets, abuse handling, and a thing that can be down. Adopt only if submission friction or "votes require GitHub" produces real, observed complaints.
7. Option C: stay dynamic but cheaper (assessed and not recommended)
For completeness: the v1 feature set could be rehosted on Fly.io/Railway/Render (~$5–15/mo) with MongoDB Atlas M0 (free) or SQLite + Litestream. This cuts the AWS bill by ~70% but keeps ~100% of the real cost — the dependency treadmill, the OAuth apps, the security surface, the backup ownership. Given the 2019 codebase needs a rewrite to be maintainable anyway, spending that rewrite on a server app rebuilds the problem. Not recommended.
8. Feature-by-feature mapping: v1 → v2
engagement.jsonviewersarray in MongoConsciously dropped: unique-authenticated-viewer counts, downvotes, the invite system (obsoleted by PR review), per-user profile pages with bios (author identity becomes the GitHub profile linked from
meta.yml; a per-author listing page is easy build-time glue if wanted).9. Migration plan
mongodump: for each non-deleted submission, writenotebook.ipynbfromnotebookJSONString(with a fallback to the archived.ipynbfiles from the instance's/filesvolume) andmeta.ymlfrom the metadata fields. TheUsercollection'sgithub.usernamefield lets you attach real GitHub identities to many authors automatically.redirects.ymlmapping old/submission/<mongo-id>paths to new slugs. On Cloudflare Pages this becomes a_redirectsfile (real 301s); on GitHub Pages, generate meta-refresh +rel=canonicalstub pages. Inbound links and search ranking survive.10. Cost comparison
(AWS figures are approximate on-demand estimates; the ops row is where the real savings live.)
11. Risks and open questions
12. Suggested next steps
The strategic shape of this rebuild: v1 was a product your team had to operate; v2 is a pipeline your team curates. Everything that used to page someone at 2 a.m. — the database, the sessions, the uploads, the email, the backups — either becomes a file in git or someone else's managed service, and the notes themselves end up faster to load, easier to find, and reviewed before they're published.