Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 80 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,52 +2,75 @@

# html2rss-web

html2rss-web converts arbitrary websites into RSS 2.0 feeds with a slim Ruby backend and a Preact frontend.
`html2rss-web` serves RSS/JSON feeds from website sources using a Ruby (Roda) backend and a Preact frontend.

## Links
## Use This Repo For

- Docs & feed directory: https://html2rss.github.io
- Contributor Guide: [docs/README.md](docs/README.md)
- Running a self-hosted `html2rss-web` instance with Docker Compose.
- Creating signed, per-account feed URLs through `POST /api/v1/feeds`.
- Local development inside the repository Dev Container.

## Quick Links

- Public docs + feed directory: https://html2rss.github.io
- Docker Hub image: https://hub.docker.com/r/html2rss/web
- OpenAPI file in this repo: [public/openapi.yaml](public/openapi.yaml)
- Contributor guide: [docs/README.md](docs/README.md)
- Discussions: https://github.com/orgs/html2rss/discussions
- Sponsor: https://github.com/sponsors/gildesmarais

## Highlights

- Responsive Preact interface for demo, sign-in, conversion, and result flows.
- Automatic source discovery with token-scoped permissions.
- Signed public feed URLs that work in standard RSS readers.
- Built-in URL validation, scoped feed access controls, and HMAC-protected tokens.
## Architecture Snapshot

## Architecture Overview
- Backend: Ruby + Roda (`app.rb`, `app/web/**`)
- Frontend: Preact + Vite (built assets served from `frontend/dist`)
- Feed extraction: `html2rss` gem
- Distribution baseline: `docker-compose.yml`

- **Backend:** Ruby + Roda, backed by the `html2rss` gem for extraction.
- **Frontend:** Preact app built with Vite into `frontend/dist` and served at `/`.
- **Distribution:** Docker Compose by default.
For detailed architecture and contributor rules, see [docs/README.md](docs/README.md).

For detailed architecture and internal rules, see [docs/README.md](docs/README.md).
## Trial Run (Docker Compose)

## Trial Run (Docker Pull And Run)
Prerequisite: Docker Engine + Docker Compose.

The published image already includes a sample `config/feeds.yml`, so you can try the app without creating or mounting one first.
Run from the repository root:

```bash
docker run --rm \
-p 4000:4000 \
-e RACK_ENV=production \
-e HTML2RSS_SECRET_KEY=$(openssl rand -hex 32) \
html2rss/web
BUILD_TAG="$(date +%F)" \
GIT_SHA="trial" \
HTML2RSS_SECRET_KEY="$(openssl rand -hex 32)" \
HEALTH_CHECK_TOKEN="$(openssl rand -hex 24)" \
BROWSERLESS_IO_API_TOKEN="trial-browserless-token" \
docker compose up -d
```

Then open:

- `http://localhost:4000/` for the web UI
- `http://localhost:4000/microsoft.com/azure-products.rss` for a built-in Azure updates feed
- `http://localhost:4000/` (UI)
- `http://localhost:4000/api/v1` (API metadata)
- `http://localhost:4000/openapi.yaml` (OpenAPI document)

Stop with:

```bash
docker compose down
```

## Deploy With Docker Compose

The checked-in [`docker-compose.yml`](docker-compose.yml) requires these environment variables for `html2rss-web`:

- `BUILD_TAG`
- `GIT_SHA`
- `HTML2RSS_SECRET_KEY`
- `HEALTH_CHECK_TOKEN`
- `BROWSERLESS_IO_API_TOKEN`

This trial run is intentionally minimal. Use Docker Compose for Browserless, auto-updates, or local feed overrides.
Optional runtime variables:

## Deploy (Docker Compose)
- `SENTRY_DSN`
- `SENTRY_ENABLE_LOGS` (defaults to `false`)

Quick start:
Example:

```bash
export HTML2RSS_SECRET_KEY="$(openssl rand -hex 32)"
Expand All @@ -56,25 +79,43 @@ export BROWSERLESS_IO_API_TOKEN="replace-with-your-browserless-token"
export BUILD_TAG="local"
export GIT_SHA="$(git rev-parse --short HEAD 2>/dev/null || echo dev)"
export AUTO_SOURCE_ENABLED=true
docker-compose up

docker compose up -d
```

Optional:
## Runtime Behavior That Affects Operations

```bash
export SENTRY_DSN="https://examplePublicKey@o0.ingest.sentry.io/0"
export SENTRY_ENABLE_LOGS=true
```
- In production, missing `HTML2RSS_SECRET_KEY` stops startup.
- `BUILD_TAG` and `GIT_SHA` are expected in production; missing values produce a startup warning.
- `POST /api/v1/feeds` requires a bearer token and only works when `AUTO_SOURCE_ENABLED=true`.
- `AUTO_SOURCE_ENABLED` defaults to `true` in development/test and `false` otherwise.
- Strategy support comes from `Html2rss::RequestService` (`faraday` and `browserless` availability is runtime-dependent).

## Stable Integration Entry Points

- OpenAPI: `/openapi.yaml` (or [`public/openapi.yaml`](public/openapi.yaml) in-repo)
- API metadata: `/api/v1`
- Feed creation endpoint: `POST /api/v1/feeds`
- Health endpoints: `/api/v1/health`, `/api/v1/health/ready`, `/api/v1/health/live`

UI + API run on `http://localhost:4000`. The app exits if the secret key is missing.
For feed config authoring/validation, use the `html2rss` schema:

## Development
- https://github.com/html2rss/html2rss/blob/master/schema/html2rss-config.schema.json
- `html2rss schema`

Use the repository's [Dev Container](.devcontainer/README.md) for all local development and tests.
Running the app directly on the host is not supported.
## Development (Dev Container Only)

All local development and test execution should run inside the repository Dev Container.

```bash
make setup
make dev
make ready
```

See the [Contributor Guide](docs/README.md) for setup commands, testing strategy, and backend structure rules.
See [docs/README.md](docs/README.md) for contributor workflows, verification gates, and architectural constraints.

## Contributing

See the [html2rss project guidelines](https://html2rss.github.io/get-involved/contributing).
- Project guidelines: https://html2rss.github.io/get-involved/contributing
- Repo contributor guide: [docs/README.md](docs/README.md)
Loading