docs: tighten strategy UX wording for end users

Author: gildesmarais

src/content/docs/creating-custom-feeds.mdx

@@ -11,14 +11,6 @@ When auto-sourcing isn't enough, you can write your own configuration files to c
 
 **Prerequisites:** You should be familiar with the [Getting Started](/getting-started) guide before diving into custom configurations.
 
-<Aside type="note" title="Release note">
-  This guide tracks the current documentation tree and may describe features that have not yet shipped in the
-  latest released `html2rss` gem. If you want the newest integrated behavior, prefer running
-  [`html2rss-web`](/web-application/getting-started) via Docker. The web application ships as a rolling
-  release and usually reflects the latest development state of the gem first. See [Versioning and
-  releases](/web-application/reference/versioning-and-releases/) for details.
-</Aside>
-
 <Aside type="tip" title="Use this guide when you need more control">
   Start with included feeds first. If your site is not covered, try [automatic feed
   generation](/web-application/how-to/use-automatic-feed-generation/) next. Reach for a custom config when you
@@ -48,7 +40,7 @@ When auto-sourcing isn't enough, you can write your own configuration files to c
 3. **Validate the config** with `html2rss validate your-config.yml`
 4. **Render the feed** with `html2rss feed your-config.yml`
 5. **Add it to `html2rss-web`** so you can use it through your normal instance
-6. **Escalate strategy when needed**: if static fetching is insufficient, switch to a JavaScript/browser-based extraction strategy
+6. **Escalate request strategy when needed**: use a browser-based rendering strategy only when troubleshooting requires it
 
 This order keeps iteration fast and makes it easier to see whether the problem is the page structure, your
 selectors, or the fetch strategy.
@@ -210,7 +202,7 @@ there.
 - **No items found?** Check your selectors with browser tools (F12) - the `items.selector` might not match the page structure
 - **Invalid YAML?** Use spaces, not tabs, and ensure proper indentation
 - **Website not loading?** Check the URL and try accessing it in your browser
-- **Missing content?** Some websites load content with JavaScript - you may need a JavaScript/browser-based extraction strategy instead of plain HTTP fetching
+- **Missing content?** Try a browser-based rendering strategy during troubleshooting
 - **Wrong data extracted?** Verify your selectors are pointing to the right elements
 
 **Need more help?** See our [comprehensive troubleshooting guide](/troubleshooting/troubleshooting) or ask in [GitHub Discussions](https://github.com/orgs/html2rss/discussions).

src/content/docs/getting-started.mdx

@@ -34,6 +34,8 @@ If you are working directly with the gem instead of `html2rss-web`, start with:
 
 <Code code={`html2rss auto https://example.com/blog`} lang="bash" />
 
+For strategy behavior and manual overrides, see the [Strategy reference](/ruby-gem/reference/strategy).
+
 If the target site is unusually redirect-heavy or needs extra follow-up requests, the CLI also supports:
 
 <Code code={`html2rss auto https://example.com/blog --max-redirects 10 --max-requests 5`} lang="bash" />

src/content/docs/ruby-gem/how-to/handling-dynamic-content.mdx

@@ -5,12 +5,14 @@ description: "Learn how to handle JavaScript-heavy websites and dynamic content
 
 import { Code } from "@astrojs/starlight/components";
 
-Some websites load their content dynamically using JavaScript. The default `html2rss` strategy might not see this content.
+Some websites load their content dynamically using JavaScript. Static fetch paths may not see this content reliably.
 
 ## Solution
 
 Use a [browser-based extraction strategy](/ruby-gem/reference/strategy) when JavaScript-heavy pages do not work with default static fetching.
 
+`browserless` is common for this workflow, and `botasaurus` is an alternate browser-based strategy when you run a Botasaurus scrape API.
+
 Keep the strategy at the top level and put request-specific options under `request`:
 
 <Code

src/content/docs/ruby-gem/reference/cli-reference.mdx

@@ -22,17 +22,16 @@ Automatically discovers items from a page and prints the generated RSS feed to s
 <Code
   code={`
   html2rss auto https://example.com/articles ; \
-  html2rss auto https://example.com/app --strategy browserless ; \
-  BOTASAURUS_SCRAPER_URL="http://localhost:4010" html2rss auto https://example.com/protected --strategy botasaurus ; \
   html2rss auto https://example.com/app --strategy browserless --max-redirects 5 --max-requests 6 ; \
+  BOTASAURUS_SCRAPER_URL="http://localhost:4010" html2rss auto https://example.com/protected --strategy botasaurus ; \
   html2rss auto https://example.com/articles --items_selector ".post-card"
 `}
   lang="bash"
 />
 
 Command: `html2rss auto URL`
 
-Default behavior uses `--strategy auto`, which tries `faraday` then `botasaurus` then `browserless`.
+Default behavior is `--strategy auto`, which tries `faraday` then `botasaurus` then `browserless`.
 
 #### URL Surface Guidance For `auto`
 
@@ -52,25 +51,29 @@ When possible, pass a direct listing/update URL instead of a top-level homepage
 
 #### Failure Outcomes You Should Expect
 
-When no extractable items are found, `auto` now classifies likely causes instead of only returning a generic message:
+When no extractable items are found, `auto` classifies likely causes instead of only returning a generic message:
 
 - `blocked surface likely (anti-bot or interstitial)`:
-  - retry with `--strategy browserless`
   - try a more specific public listing URL
 - `app-shell surface detected`:
-  - retry with `--strategy browserless`
   - switch to a direct listing/update URL
 - `unsupported extraction surface for auto mode`:
   - switch to listing/changelog/category URLs
   - use explicit selectors in a feed config
 
 Known anti-bot interstitial responses (for example Cloudflare challenge pages) are surfaced explicitly as blocked-surface errors.
 
-If you run with the default `--strategy auto`, no manual strategy override is required for fallback ordering.
+If all fallback tiers run but still extract zero items, html2rss raises:
+
+- `No RSS feed items extracted after auto fallback ...`
+
+If failures continue after URL/surface fixes, retry with an explicit browser-based override (`--strategy browserless`), or `--strategy botasaurus` when `BOTASAURUS_SCRAPER_URL` is configured.
+
+Start by changing the input URL to a direct listing/update page, then move to explicit selectors if needed.
 
 #### Browserless Setup And Diagnostics (CLI)
 
-`browserless` is opt-in for CLI usage.
+`browserless` is an explicit override for CLI usage.
 
 <Code
   code={`
@@ -104,7 +107,7 @@ For custom Browserless endpoints, `BROWSERLESS_IO_API_TOKEN` is required.
 
 #### Botasaurus Environment Requirement (CLI)
 
-`botasaurus` is opt-in for CLI usage and requires `BOTASAURUS_SCRAPER_URL`:
+`botasaurus` is an explicit override for CLI usage and requires `BOTASAURUS_SCRAPER_URL`:
 
 <Code
   code={`
@@ -128,6 +131,7 @@ Loads a YAML config, builds the feed, and prints the RSS XML to stdout.
   code={`
   html2rss feed single.yml ; \
   html2rss feed feeds.yml my-first-feed ; \
+  html2rss feed single.yml --strategy auto ; \
   html2rss feed single.yml --strategy browserless ; \
   BOTASAURUS_SCRAPER_URL="http://localhost:4010" html2rss feed single.yml --strategy botasaurus ; \
   html2rss feed single.yml --max-redirects 5 --max-requests 6 ; \

src/content/docs/ruby-gem/reference/strategy.mdx

@@ -1,21 +1,26 @@
 ---
 title: Strategy
-description: "Learn about different strategies for fetching website content with html2rss. Choose between faraday, browserless, and botasaurus strategies for optimal performance."
+description: "Learn how html2rss chooses request strategies by default with auto fallback, and when to override with faraday, botasaurus, or browserless."
 ---
 
 import { Code } from "@astrojs/starlight/components";
 
 The `strategy` key defines how `html2rss` fetches a website's content.
 
-- **`faraday`** (default): Makes a direct HTTP request. It is fast but does not execute JavaScript.
+- **`auto`** (default): Tries concrete strategies in order: `faraday` -> `botasaurus` -> `browserless`.
+- **`faraday`**: Makes a direct HTTP request. It is fast but does not execute JavaScript.
 - **`browserless`**: Renders the website in a headless Chrome browser, which is necessary for JavaScript-heavy sites.
 - **`botasaurus`**: Delegates fetching to a Botasaurus scrape API. This is opt-in and requires `BOTASAURUS_SCRAPER_URL`.
 
 `strategy` is a top-level config key. Request-specific controls live under `request`.
 
-If you use CLI `--strategy auto` (default), html2rss tries `faraday` then `botasaurus` then `browserless`.
+`auto` falls back to the next strategy when the current attempt errors or extracts zero items. Use explicit `--strategy ...` only when you need to force a specific transport for troubleshooting or reproducibility.
 
-Use `faraday` first for direct newsroom/listing/changelog pages. Prefer `botasaurus` as the first explicit browser-based strategy when you have a Botasaurus scrape API. Use `browserless` when you specifically need Browserless preload actions.
+## `auto` (default)
+
+The default strategy chain is:
+
+`faraday` -> `botasaurus` -> `browserless`
 
 ## `browserless`
 
@@ -65,7 +70,7 @@ Set the `strategy` at the top level of your feed configuration and put request c
 
 Use this split consistently:
 
-- `strategy`: selects `faraday`, `browserless`, or `botasaurus`
+- `strategy`: selects `auto`, `faraday`, `browserless`, or `botasaurus`
 - `headers`: top-level headers shared by all strategies
 - `request.max_redirects`: redirect limit for the request session
 - `request.max_requests`: total request budget for the whole feed build

src/content/docs/troubleshooting/troubleshooting.mdx

@@ -32,34 +32,34 @@ The `auto` flow is URL-surface sensitive.
 
 If extraction quality is poor, switch to a more specific listing/update URL before tuning selectors.
 
-If you use CLI defaults, `--strategy auto` is already active and attempts `faraday` then `botasaurus` then `browserless`.
-
 ### Empty Feeds
 
 If your feed is empty, check the following:
 
 - **URL:** Ensure the `url` in your configuration is correct and accessible.
 - **`items.selector`:** Verify that the `items.selector` matches the elements on the page.
 - **Website Changes:** Websites change their HTML structure frequently. Your selectors may be outdated.
-- **JavaScript Content:** If the content is loaded via JavaScript, move from `faraday` to a rendering strategy such as `browserless` (or `botasaurus` when you use a Botasaurus scrape API).
+- **JavaScript Content:** If the content is loaded via JavaScript, use a browser-based rendering strategy.
 - **Authentication:** Some sites require authentication — check if you need to add headers or use a different strategy.
 
 ### `No scrapers found` Failure Taxonomy (`auto`)
 
 `auto` classifies no-scraper failures with actionable hints:
 
 - **Blocked surface likely (anti-bot or interstitial):**
-  - retry with `--strategy browserless`
   - try a more specific public listing URL
 - **App-shell surface detected:**
-  - retry with `--strategy browserless`
   - target a direct listing/update page instead of homepage/shell entrypoint
 - **Unsupported extraction surface for auto mode:**
   - switch to listing/changelog/category URLs
   - or use explicit selectors in YAML config
 
 Known anti-bot interstitial patterns (for example Cloudflare challenge pages) are surfaced as blocked-surface errors instead of silent empty extraction results.
 
+When all auto fallback tiers complete but still extract zero items, html2rss raises `No RSS feed items extracted after auto fallback ...`.
+
+If failures continue after URL/surface fixes, retry with an explicit browser-based override (`--strategy browserless`), or `--strategy botasaurus` when `BOTASAURUS_SCRAPER_URL` is configured.
+
 ### Browserless Connection / Setup Failures
 
 If you receive `Browserless connection failed (...)`:
@@ -93,7 +93,7 @@ For custom websocket endpoints, `BROWSERLESS_IO_API_TOKEN` is required.
 Common configuration-related errors:
 
 - **`UnsupportedResponseContentType`:** The website returned content that html2rss can't parse (not HTML or JSON).
-- **`UnsupportedStrategy`:** The specified strategy is not available. Use `faraday`, `browserless`, or `botasaurus`.
+- **`UnsupportedStrategy`:** The specified strategy is not available. Use `auto`, `faraday`, `browserless`, or `botasaurus`.
 - **`BOTASAURUS_SCRAPER_URL is required for strategy=botasaurus.`:** Set `BOTASAURUS_SCRAPER_URL` to your Botasaurus scrape API base URL when using `--strategy botasaurus`.
 - **`BOTASAURUS_SCRAPER_URL is invalid`:** Fix the URL format and retry.
 - **`Configuration must include at least 'selectors' or 'auto_source'`:** You need to specify either manual selectors or enable auto-source.

src/content/docs/web-application/how-to/use-automatic-feed-generation.mdx

@@ -42,7 +42,7 @@ Then restart the stack:
 1. Open your instance at `http://localhost:4000`
 2. Paste a page URL into `Create a feed`
 3. Add a valid access token when prompted
-4. Choose a strategy if needed, then submit
+4. Submit the request
 5. Copy the generated feed URL or open it directly
 
 ## What Success Looks Like
@@ -59,23 +59,21 @@ That is enough to confirm the self-hosted flow is working.
 
 ## Strategy Behavior
 
-- `faraday` is the default strategy and should be your first try for most pages.
-- During the feed-creation API request (`POST /api/v1/feeds`) from the web UI, a `faraday` submission may be retried once with `browserless` when the first failure looks retryable.
-- If that fallback attempt fails, or if the first failure is clearly auth/URL/unsupported-strategy related, the UI stops and shows an error.
-- This retry behavior is scoped to feed creation. It is not a general retry layer for later feed rendering (`GET /api/v1/feeds/:token`) or preview loading.
+- Feed creation uses the backend default strategy behavior.
+- If feed creation fails, the UI surfaces structured retry/error guidance rather than exposing low-level strategy controls.
 
 ## Input URL Guidance (Quality First)
 
 Automatic generation is most successful when the input URL is already a listing/update surface.
 
 - Higher-success inputs:
-- newsroom/press listing pages
-- category/tag/archive/listing pages
-- changelog/release/update pages
+  - newsroom/press listing pages
+  - category/tag/archive/listing pages
+  - changelog/release/update pages
 - Lower-success inputs:
-- generic homepages
-- search pages
-- app-shell entrypoints (client-rendered shells)
+  - generic homepages
+  - search pages
+  - app-shell entrypoints (client-rendered shells)
 
 If output quality is poor, switch the input to a direct listing/update URL before assuming the feature is broken.