diff --git a/concepts/sandboxes.mdx b/concepts/sandboxes.mdx index ba34567..8468d39 100644 --- a/concepts/sandboxes.mdx +++ b/concepts/sandboxes.mdx @@ -8,7 +8,7 @@ description: "Run pipeline steps and module builds in Firecracker microVMs on an A sandbox runs one pipeline step or module build in its own [Firecracker](https://firecracker-microvm.github.io/) microVM, on EC2 hosts in your AWS account. -You set the CPU, memory and disk a step needs. Ravion launches, packs, scales and retires the hosts. On a running host, a sandbox boots in one to two seconds, and the [disk cache](#disk-cache) brings back the whole disk from the last successful run. +You set the CPU, memory and disk a step needs. Ravion launches, packs, scales and retires the hosts. On a running host, a sandbox boots in one to two seconds, and the [disk cache](#disk-cache) brings back the whole disk from the last saved run. ## How the fleet works @@ -111,11 +111,11 @@ For faster starts after quiet periods, you can keep a few hosts on hand: running ## Why sandboxes are fast -A cold CI job spends much of its time getting back to where the last one was: installing tools, restoring caches and pulling Docker layers. A sandbox skips that. It boots in one to two seconds with the whole disk from the last successful run already attached. +A cold CI job spends much of its time getting back to where the last one was: installing tools, restoring caches and pulling Docker layers. A sandbox skips that. It boots in one to two seconds with the whole disk from the last saved run already attached. ### The whole disk is the cache -When `disk_cache` is on, Ravion saves the sandbox's entire root disk after a successful run. You don't list cache paths. Everything the step left behind comes back next time: +When `disk_cache` is on, Ravion saves the sandbox's entire root disk when a step succeeds or one of its commands fails. You don't list cache paths. Everything the step left behind comes back next time: - Packages installed with `apt`, `setup` actions or install scripts - `node_modules`, `~/.cache`, `~/.m2`, Go and Cargo build caches, `.next/cache`, `.turbo` @@ -134,7 +134,7 @@ The size of the disk doesn't change the boot time, and the step rarely waits on ### Saves upload only what changed -While the step runs, the host uploads changed chunks in the background. When the step succeeds, only the remaining chunks are left to upload. Unchanged chunks are never uploaded again, and empty chunks cost nothing. +While the step runs, the host uploads changed chunks in the background. When the step succeeds or one of its commands fails, only the remaining chunks are left to upload. Ravion does not save a disk when a step is cancelled, times out, runs out of disk, or loses its host. Unchanged chunks are never uploaded again, and empty chunks cost nothing. ```mermaid sequenceDiagram @@ -147,7 +147,7 @@ sequenceDiagram Store-->>Host: Prefetch what the last run read VM->>Host: Read blocks (served from NVMe cache) Host->>Store: Upload changed chunks while the step runs - VM->>Host: Step succeeds + VM->>Host: Step succeeds or a command fails Host->>Store: Upload the rest and save for the branch ``` @@ -355,9 +355,9 @@ Pick a region close to your data instead when: | --- | --- | | `off` | Start from the base image every run. Default for pipeline steps. | | `auto` | Keep this step's own disk, named by its git source and step. Needs a git `source`. | -| A key, such as `web-deps` | Share one disk across every step in the repository that uses the key. The last successful run to finish saves it. Keys are up to 64 lowercase letters, digits, `.`, `_` or `-`. | +| A key, such as `web-deps` | Share one disk across every step in the repository that uses the key. The last eligible run to finish saves it. Keys are up to 64 lowercase letters, digits, `.`, `_` or `-`. | -A run restores the disk saved for its branch. If there's none, it uses the pull request's base branch, then the repository's default branch. A run saves its disk to its own branch only when it succeeds. +A run restores the disk saved for its branch. If there's none, it uses the pull request's base branch, then the repository's default branch. A step saves its disk to its own branch when it succeeds or one of its commands fails. It does not save when cancelled, timed out, out of disk, or when its host is lost. To start a run on clean disks, pass `--no-cache` to `ravion pipeline run create` or `ravion pipeline run retry`. @@ -378,6 +378,14 @@ New sandbox jobs resume after cleanup completes. If cleanup fails, they stay pau Deleting caches cannot be undone. Fix or remove a compromised dependency before running jobs again; deleting its cached files does not make the dependency safe or revoke exposed credentials. +## Shared build cache for pipeline steps + +Private pipeline sandbox steps with a [git source](#pipeline-steps) automatically use a shared build cache, independently of `disk_cache`. Ravion configures Go builds through `GOCACHEPROG`. If you already use [sccache](https://github.com/mozilla/sccache), Rust builds use the shared cache through `SCCACHE_WEBDAV_ENDPOINT`. + +Ravion scopes cache entries to the repository and branch, and reads from the repository's default branch when the branch has no entry. Pipeline steps share entries with that repository's [GitHub Actions jobs](/github-actions/caching). + +Your own `GOCACHEPROG` or sccache settings take precedence. To turn off either cache for a step, set `RAVION_GO_CACHE_PROG=off` or `RAVION_SCCACHE=off` in its environment. The shared build cache does not apply inside [container instructions](#what-runs-inside-a-sandbox) or Docker builds. + ## Limits and caveats - **Account and region targets.** With `aws_account_id` and `region`, Ravion uses that account and region's execution environment, and creates one in the default VPC if none exists. Use `execution_environment_id` to choose the network. diff --git a/github-actions/caching.mdx b/github-actions/caching.mdx index 5ba94fe..da65f98 100644 --- a/github-actions/caching.mdx +++ b/github-actions/caching.mdx @@ -72,6 +72,8 @@ Ravion doesn't add the flag for you, because Bazel fails when an HTTP cache is c If the cache is unreachable, these tools build without it. Turborepo and Nx remote caches aren't filled in, because their cache keys can miss inputs you haven't declared, such as environment variables, and a shared cache could then serve another branch's output. +Pipeline sandbox steps with a git source use the same cache for Go and Rust builds; see [shared build cache for pipeline steps](/concepts/sandboxes#shared-build-cache-for-pipeline-steps). + Public repositories get a separate cache per repository and can't read private caches. EC2 runners use GitHub's cache. ## Artifacts diff --git a/modules/build.mdx b/modules/build.mdx index 4b98814..642afdb 100644 --- a/modules/build.mdx +++ b/modules/build.mdx @@ -49,7 +49,7 @@ Add a `build` step to your pipeline and point it at the module instance: - `module_instance` accepts a unique ID (`mi_…`), `environment.module`, or `project.environment.module`. - `input` passes build-time values the module accepts, such as a tag, branch, or commit. Check the module's docs or schema (`ravion module schema `) for what it takes. -Builds run in your AWS account, so your source code never enters the Ravion control plane. Sandbox builds keep [Docker layer cache](#docker-layer-cache) on disk between successful runs. +Builds run in your AWS account, so your source code never enters the Ravion control plane. Sandbox builds keep [Docker layer cache](#docker-layer-cache) on disk after a build succeeds or one of its commands fails. Building something that isn't a module? Use the standalone `build:image` or `build:static` [CI @@ -66,13 +66,13 @@ The build uses the module's Terraform execution environment unless you set **Bui Existing module instances keep their saved builder settings when upgraded. To move an existing EC2 build to a sandbox, open **Builder config**, set **Builder capacity type** to **Sandbox**, and choose its CPU and memory. -Sandbox builds use `disk_cache: auto` to keep dependencies, build output and Docker layers between successful runs. **Interruptible build** is on by default, allowing spot hosts and a restart from the beginning if the host is lost. Turn it off if your build changes external state, such as running [database migrations](/guides/database-migrations). +Sandbox builds use `disk_cache: auto` to keep dependencies, build output and Docker layers after a build succeeds or one of its commands fails. **Interruptible build** is on by default, allowing spot hosts and a restart from the beginning if the host is lost. Turn it off if your build changes external state, such as running [database migrations](/guides/database-migrations). See [Sandboxes](/concepts/sandboxes) for pool settings, disk cache and cold-start behavior. ## Docker layer cache -Sandbox module builds keep Docker's build cache on the sandbox disk. With `disk_cache: auto`, Ravion saves the disk after a successful run and restores it for later runs. Dependencies, build output and Docker layers remain available without configuring a separate cache. See [Disk cache](/concepts/sandboxes#disk-cache) for branch restore rules. +Sandbox module builds keep Docker's build cache on the sandbox disk. With `disk_cache: auto`, Ravion saves the disk when a build succeeds or one of its commands fails, then restores it for later runs. Dependencies, build output and Docker layers remain available without configuring a separate cache. See [Disk cache](/concepts/sandboxes#disk-cache) for branch restore rules. Disk caching also benefits static asset and Lambda builds. diff --git a/styles/config/vocabularies/Mintlify/accept.txt b/styles/config/vocabularies/Mintlify/accept.txt index b8f4126..c7efa79 100644 --- a/styles/config/vocabularies/Mintlify/accept.txt +++ b/styles/config/vocabularies/Mintlify/accept.txt @@ -103,6 +103,7 @@ rollouts routing_policy runbooks? scaffolded +sccache semver serverless set_identifier