Skip to content

docs: consolidate contributor and architecture guidance #897

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gildesmarais merged 3 commits into from
Mar 21, 2026
Merged

docs: consolidate contributor and architecture guidance #897

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b7f7a9a
docs: consolidate contributor and architecture guidance
gildesmarais Mar 21, 2026
c6ab4c1
Fix documentation links and renderer references
gildesmarais Mar 21, 2026
0915f32
Convert architecture diagram to Mermaid
gildesmarais Mar 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 10 additions & 69 deletions .github/copilot-instructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,75 +1,16 @@
# html2rss-web AI Agent Instructions

## Overview
This repository uses a centralized documentation structure. All AI agents (including Copilot, Gemini CLI, etc.) must follow the constraints and rules defined in the following files:

- Ruby web app that converts websites into RSS 2.0 feeds.
- Built with **Roda** backend + **Preact** frontend, using the **html2rss** gem (+ `html2rss-configs`).
- **Frontend:** Vite-built Preact UI, served alongside Ruby backend.
## Canonical Documentation

## Documentation website of core dependencies
- **Agent Constraints**: [AGENTS.md](../../AGENTS.md) (Execution rules, verification, and UI principles)
- **Contributor Guide**: [docs/README.md](../../docs/README.md) (Architecture, security, setup, and coding style)
- **Design System**: [docs/design-system.md](../../docs/design-system.md) (Visual and CSS rules)
Comment thread
gildesmarais marked this conversation as resolved.
Outdated

Search these pages before using them. Find examples, plugins, UI components, and configuration options.
## Quick Reference for Agents

### Roda

1. https://roda.jeremyevans.net/documentation.html

### Preact & Vite

1. https://preactjs.com/guide/v10/getting-started/
2. https://vite.dev/guide/

### html2rss

1. If available, find source locally in: `../html2rss`.
2. source code on github: https://github.com/html2rss/html2rss

### Test and Linters

1. https://docs.rubocop.org/rubocop/cops.html
2. https://docs.rubocop.org/rubocop-rspec/cops_rspec.html
3. https://rspec.info/features/3-13/rspec-expectations/built-in-matchers/
4. https://www.betterspecs.org/

Fix rubocop `RSpec/MultipleExpectations` adding rspec tag `:aggregate_failures`.

## Core Rules

- ✅ Organise Roda routes via dedicated modules (e.g. `Html2rss::Web::Routes::*`), keeping the main app class thin.
- ✅ Keep helper modules minimal: define entrypoints with `class << self` and push implementation helpers under `private`; avoid `module_function` unless mirroring existing conventions.
- ✅ Validate all inputs. Pass outbound requests through **SSRF filter**.
- ✅ Add caching headers where appropriate (`Rack::Cache`).
- ✅ Errors: friendly messages for users, detailed logging internally.
- ✅ **Frontend**: Use Preact components in `frontend/src/`. Keep it simple.
- ✅ **CSS**: Use the app-owned frontend styles in `frontend/src/styles/`.
- ✅ **Specs**: RSpec for Ruby, build tests for frontend.
- ✅ When a spec needs to tweak environment variables, wrap the example in `ClimateControl.modify` so state is restored automatically.

## Don't

- ❌ Don't use Ruby's URI class or addressable gem directly. Strictly use `Html2rss::Url` only.
- ❌ Don't bypass SSRF filter or weaken CSP.
- ❌ Don't add databases, ORMs, or background jobs.
- ❌ Don't leak stack traces or secrets in responses.
- ❌ Don't reach into private API with `send(...)`; expose what you need at the module level instead.
- ❌ Don't modify `frontend/dist/` - it's generated by build process.
- ❌ NEVER expose the auth token a user provides.

## Environment

- `RACK_ENV` – environment
- `AUTO_SOURCE_ENABLED`, `AUTO_SOURCE_USERNAME`, `AUTO_SOURCE_PASSWORD`, `AUTO_SOURCE_ALLOWED_ORIGINS`
- `HEALTH_CHECK_USERNAME`, `HEALTH_CHECK_PASSWORD`
- `SENTRY_DSN` (optional)

### Verification Steps

- Run `ruby -c app.rb` to check syntax
- Run `bundle exec rspec` to verify tests
- Check `bundle install` removes unused dependencies

## Style

- Add `# frozen_string_literal: true`
- Follow RuboCop style
- YARD doc comments for public methods
- **Environment**: All commands MUST run inside the Dev Container.
- **Verification**: Run `make ready` before any commit.
- **Security**: Follow strict [Security & Safety Rules](../../docs/README.md#security--safety-rules).
- **Style**: Follow [Architectural Constraints](../../docs/README.md#architectural-constraints) (YARD docs, Roda organization).
Comment thread
gildesmarais marked this conversation as resolved.
Outdated
101 changes: 34 additions & 67 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,84 +1,51 @@
# Agent Workflow (Dev Container)
# Agent Workflow Constraints

## Start the Dev Container

```text
docker compose -f .devcontainer/docker-compose.yml up -d
```

## Commands (run inside the container)

```text
docker compose -f .devcontainer/docker-compose.yml exec -T app make setup

docker compose -f .devcontainer/docker-compose.yml exec -T app make dev

docker compose -f .devcontainer/docker-compose.yml exec -T app make test

docker compose -f .devcontainer/docker-compose.yml exec -T app make ready

docker compose -f .devcontainer/docker-compose.yml exec -T app bundle exec rubocop -F

docker compose -f .devcontainer/docker-compose.yml exec -T app bundle exec rspec
```

Pre-commit gate (required):

```text
docker compose -f .devcontainer/docker-compose.yml exec -T app make ready
```

If you need an interactive shell:

```text
docker compose -f .devcontainer/docker-compose.yml exec app bash
```

---
This document defines execution constraints for AI agents. For general contributor rules, setup commands, architectural constraints, and security policies, see [docs/README.md](docs/README.md).

## Collaboration Agreement (Agent ↔ User)

## Interview Answers (ID-able) + Expert Recommendations

**DoD:** `make ready` in Dev Container; if applicable, user completes manual smoke test with agent-provided steps.
**Verification:** Always smoke Dev Container + `make ready`.
**Commits:** Group by logical unit after smoke-tested (feature / improvement / refactor).
**Responses:** Changes → Commands → Results → Next steps, ending with a concise one-line summary.
**KISS vs Refactor:** KISS by default; boy-scout refactors allowed if low-risk and simplifying.
**Ambiguity:** Proceed with safest assumption, then confirm.
**Non-negotiables:** Dev Container only; security first.

Expert recommendation: keep workflows terminal-first and keyboard-focused (clear commands, no GUI-only steps).
- **DoD:** `make ready` in Dev Container; if applicable, user completes manual smoke test with agent-provided steps.
- **Verification:** Always smoke Dev Container + `make ready`.
- **Commits:** Group by logical unit after smoke-tested (feature / improvement / refactor).
- **Responses:** Changes → Commands → Results → Next steps, ending with a concise one-line summary.
- **KISS vs Refactor:** KISS by default; boy-scout refactors allowed if low-risk and simplifying.
- **Ambiguity:** Proceed with safest assumption, then confirm.
- **Non-negotiables:** Dev Container only; security first.

## Definition of Done

- Run `make ready` inside the Dev Container.
- If applicable, user completes manual smoke test; agent provides clear instructions.

## Verification Rules
## Agent-Specific Verification Rules

- Always run Dev Container smoke + `make ready` for changes.
- For frontend changes, also verify in `chrome-devtools` MCP at `http://127.0.0.1:4001/` while the Dev Container is running.
- Capture a quick state check for all affected UI states (e.g., guest/member/result) to enforce state parity and avoid duplicate actions.

## Commit Granularity
### Frontend Smoke Checklist

- Group commits by logical units after they have grown and been smoke-tested (feature / improvement / refactor).
- Start the Dev Container and app (`make dev`).
- Open `http://127.0.0.1:4001/` with `chrome-devtools` MCP.
- Validate the primary user path touched by the change.
- Verify all affected states (e.g., guest/member/result) keep the same layout grammar.
- Confirm action uniqueness: one canonical control per outcome in each state.

## Response Format
## UI Execution Principles

- Default: Changes → Commands → Results → Next steps.
- End with a concise one-line summary.
See [docs/design-system.md](docs/design-system.md) for visual rules.

## KISS vs Refactor
- **Task Dominance:** Each UI state should make one user objective obvious and primary. Supporting surfaces and links must yield priority.
- **Copy Minimalism:** Remove text that repeats what the interface already communicates. Prefer action-oriented wording.
- **State Skeleton:** Adjacent UI states should read as the same frame with content changes, not as separate pages.
- **Focus Contract:** Verify browser autofocus and return-focus behavior on initial load and after transitions.
- **Support Compression:** When a user has advanced past setup, reduce the visual weight of support content.

- KISS by default.
- Boy-scout refactors are allowed when they reduce complexity and are low-risk.

## Ambiguity Handling
## Response Format

- Proceed with the safest assumption, then confirm.
1. **Changes:** Briefly list files/symbols modified.
2. **Commands:** Show the verification commands run.
3. **Results:** Summarize the outcome.
4. **Next steps:** Propose the immediate follow-up.
5. **One-line Summary:** End with a concise summary.

## Non-Negotiables

- Security first.
- YARD docs are strict for public Ruby methods in `app/`: every public method must have a YARD docstring with typed `@param` tags (for all params) and typed `@return`.
- When touching non-public methods, add YARD docs when it improves maintenance or clarifies invariants/edge handling.
- **Security first:** No leaking secrets or insecure patterns. See [Security & Safety Rules](docs/README.md#security--safety-rules).
- **YARD docs:** Strict for public Ruby methods in `app/`. Every public method must have a YARD docstring with typed `@param` and `@return`. See [Architectural Constraints](docs/README.md#architectural-constraints).
- **No host execution:** All commands MUST run inside the Dev Container via `make` or `bundle exec`.
100 changes: 23 additions & 77 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ html2rss-web converts arbitrary websites into RSS 2.0 feeds with a slim Ruby bac
## Links

- Docs & feed directory: https://html2rss.github.io
- Contributor Guide: [docs/README.md](docs/README.md)
- Discussions: https://github.com/orgs/html2rss/discussions
- Sponsor: https://github.com/sponsors/gildesmarais

Expand All @@ -15,29 +16,35 @@ html2rss-web converts arbitrary websites into RSS 2.0 feeds with a slim Ruby bac
- Responsive Preact interface for demo, sign-in, conversion, and result flows.
- Automatic source discovery with token-scoped permissions.
- Signed public feed URLs that work in standard RSS readers.
- Built-in SSRF defences, input validation, and HMAC-protected tokens.
- Built-in URL validation, scoped feed access controls, and HMAC-protected tokens.

## Architecture
## Architecture Overview

- **Backend:** Ruby + Roda, backed by the `html2rss` gem for extraction.
- **Frontend:** Preact app built with Vite into `frontend/dist` and served at `/`.
- **Distribution:** Docker Compose by default; other deployments require manual wiring.
- [Project notes](docs/README.md)
- **Distribution:** Docker Compose by default.

## REST API Snapshot
For detailed architecture and internal rules, see [docs/README.md](docs/README.md).

## Trial Run (Docker Pull And Run)

The published image already includes a sample `config/feeds.yml`, so you can try the app without creating or mounting one first.

```bash
# List available strategies
curl -H "Authorization: Bearer <token>" \
"https://your-domain.com/api/v1/strategies"

# Create a feed and capture the signed public URL
curl -X POST "https://your-domain.com/api/v1/feeds" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","name":"Example Feed"}'
docker run --rm \
-p 4000:4000 \
-e RACK_ENV=production \
-e HTML2RSS_SECRET_KEY=$(openssl rand -hex 32) \
html2rss/web
```

Then open:

- `http://localhost:4000/` for the web UI
- `http://localhost:4000/microsoft.com/azure-products.rss` for a built-in Azure updates feed

This trial run is intentionally minimal. Use Docker Compose for Browserless, auto-updates, or local feed overrides.

## Deploy (Docker Compose)

1. Generate a key: `openssl rand -hex 32`.
Expand All @@ -46,73 +53,12 @@ curl -X POST "https://your-domain.com/api/v1/feeds" \

UI + API run on `http://localhost:4000`. The app exits if the secret key is missing.

## Development (Dev Container)
## Development

Use the repository's [Dev Container](.devcontainer/README.md) for all local development and tests.
Running the app directly on the host is not supported.

Quick start inside the Dev Container:

```
make setup
make dev
make test
make ready
make yard-verify-public-docs
bundle exec rubocop -F
bundle exec rspec
make openapi
```

Dev URLs: Ruby app at `http://localhost:4000`, frontend dev server at `http://localhost:4001`.

Backend code under the `Html2rss::Web` namespace now lives under `app/web/**`, so Zeitwerk can mirror constant paths directly instead of relying on directory-specific namespace wiring.
`make ready` also runs `rake zeitwerk:verify`, which eager-loads the app and fails on loader drift early.
For contributors and AI agents changing backend structure, follow the rules in [docs/README.md](docs/README.md).

## Make Targets

| Command | Purpose |
| ------------------------------ | ---------------------------------------------------------- |
| `make help` | List available shortcuts. |
| `make setup` | Install Ruby and Node dependencies. |
| `make dev` | Run Ruby (port 4000) and frontend (port 4001) dev servers. |
| `make dev-ruby` | Start only the Ruby server. |
| `make dev-frontend` | Start only the frontend dev server (port 4001). |
| `make test` | Run Ruby and frontend test suites. |
| `make test-ruby` | Run Ruby specs. |
| `make test-frontend` | Run frontend unit and contract tests. |
| `make lint` | Run all linters. |
| `make lintfix` | Auto-fix lint warnings where possible. |
| `make yard-verify-public-docs` | Enforce typed YARD docs for public methods in `app/`. |
| `make openapi` | Regenerate `public/openapi.yaml` from request specs. |
| `make openapi-verify` | Regenerate + fail if OpenAPI file is stale. |
| `make clean` | Remove build artefacts. |

## OpenAPI Contract

The OpenAPI file is generated from Ruby request specs only.

- Regenerate: `make openapi`
- Verify drift (CI behavior): `make openapi-verify`

## Frontend npm Scripts (inside Dev Container)

| Command | Purpose |
| ----------------------- | -------------------------------------------- |
| `npm run dev` | Vite dev server with hot reload (port 4001). |
| `npm run build` | Build static assets into `frontend/dist/`. |
| `npm run test:run` | Unit tests (Vitest). |
| `npm run test:contract` | Contract tests with MSW. |

## Testing Strategy

| Layer | Tooling | Focus |
| ----------------- | ------------------------ | ---------------------------------------------------- |
| Ruby API | RSpec + Rack::Test | Feed creation, retrieval, auth paths. |
| Frontend unit | Vitest + Testing Library | Component rendering and hooks with mocked fetch. |
| Frontend contract | Vitest + MSW | End-to-end fetch flows against mocked API responses. |
| Docker smoke | RSpec (`:docker`) | Net::HTTP probes against the containerised service. |
See the [Contributor Guide](docs/README.md) for setup commands, testing strategy, and backend structure rules.

## Contributing

Expand Down
Loading
Loading