Skip to content
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion architecture/0001-npm-lockfiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Date: 2020-03-02

## Status

Accepted
Superseded 2021-10-09 by decision to use yarn everywhere.

## Context

Expand Down
56 changes: 56 additions & 0 deletions architecture/0002-docker-multi-arch-build.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# 2. Docker multi-architecture build using create-docker-context

Date: 2024-07-18

## Status

Proposed

## Context

I (crispy) consider a multi-stage dockerfile to be the gold standard of
reproducible, mutli-architecture builds. Something like the following:

- Build container copies workspace, installs development dependencies and builds
the app.
- Production container copies build artifacts and installs only production
dependencies.

is ideal. This ensures only production dependencies are present, and you can run
this process on every architecture to create a multi-arch docker image. Binaries
downloaded during dependency installation will fetch the correct architecture
binary as depencies are installed separately on each architecture. But
installing dependencies and building JS is extremely slow on emulated
architectures, such as docker buildx on GitHub Actions (this can take 2.5 hours
to build the image).

If instead we can (on the build machine/VM):

1. install all dependencies
2. build the app
3. copy build artifacts and only production dependencies to the multi-arch
docker image

then **as long as production dependencies are portable**, we have very little
computation being run on emulated architectures. The `create-docker-context.js`
script allows us to do this, copying build artifacts and only production
dependencies to an intermediate "context" folder which is then used to create
the

Currently none of our production dependencies install non-portable binaries.

## Decision

While all production dependencies remain portable, we will build
multi-architecture docker images by building TerriaMap on the VM and copying
only production-necessary files and dependencies to the final docker image.

## Consequences

- We will replace current GitHub Actions release process with one using
`create-docker-context.js`.
- Our GitHub Actions TerriaMap release time will reduce from 2.5 hours to less
than 10 minutes.
- If in future TerriaMap uses a binary installed by side effect during JS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is a big deal, Github provides ARM runners now so if this becomes a problem in the future it's "easy" to rectify.

dependency installation and this binary cannot be run on an architecture for
which an image is created, that image will fail when run on that architecture.
10 changes: 9 additions & 1 deletion deploy/docker/create-docker-context.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
#!/usr/bin/env node

// MAJOR ASSSUMPTION: build artifacts and node_modules content for all production
// dependencies is cross-platform, or care is taken to only install dependencies,
// run create-docker-context.js and create docker images on a compatible platform
// See architecture/0002-docker-multi-arch-build.md

// Based off @magda/docker-utils@2.1.0 create-docker-context-for-node-component
// Changes made:
// - The Dockerfile path is configurable in package.json
// - The Dockerfile path is configurable in package.json (I don't want a dockerfile
// intended to be used only through a script to be in the top level directory)
// - Can parse metadata from GitHub Action docker/metadata-action@v5 and add this
// to the created image

const childProcess = require("child_process");
const fse = require("fs-extra");
Expand Down