Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
*
!extensions/
!nvlib/
!patches/
!prebuilt/
!rapt/
!renios/
!renpybuild/
!runtime/
!source/
!specs/
!steamapi/
!tasks/
!tools/
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.sh text eol=lf
37 changes: 37 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
FROM ubuntu:24.04

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
rm -f /etc/apt/apt.conf.d/docker-clean && \
apt-get update && apt-get install --fix-missing --no-install-recommends -y \
git build-essential ccache unzip autoconf autoconf-archive automake libtool-bin \
python3-dev python3-pip python3-venv \
curl \
python3-jinja2 \
debootstrap qemu-user-static \
libgmp-dev libmpfr-dev libmpc-dev \
software-properties-common \
libssl-dev libbz2-dev liblzma-dev \
bc \
cmake clang libxml2-dev llvm \
quilt \
ninja-build \
libavcodec-dev libavformat-dev \
libswresample-dev libswscale-dev libfreetype6-dev libfribidi-dev libsdl2-dev \
libsdl2-image-dev libsdl2-gfx-dev libsdl2-mixer-dev libsdl2-ttf-dev libjpeg-dev \
libharfbuzz-dev libassimp-dev

RUN wget -O /tmp/llvm.sh https://apt.llvm.org/llvm.sh && \
chmod +x /tmp/llvm.sh && \
/tmp/llvm.sh 18 && \
rm /tmp/llvm.sh

COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/

RUN uv python install 3.12

RUN --mount=type=bind,source=prebuilt/clang_rt.tar.gz,target=/tmp/clang_rt.tar.gz \
mkdir -p /usr/lib/llvm-18/lib/clang/18/lib/ && \
tar xzf /tmp/clang_rt.tar.gz -C /usr/lib/llvm-18/lib/clang/18/lib/

COPY . /build
181 changes: 105 additions & 76 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,118 +6,147 @@ can build the binary components of Ren'Py and all its dependencies, in
the same manner that is used to make official Ren'Py releases.

Requirements
-------------
------------

Ren'Py Build requires a computer running Ubuntu 24.04. While it can run on
a desktop computer, portions of the build process must run at root, and the
whole process has security implications. My recommendation is to create a
virtual machine, install Ubuntu 24.04 on it, and run this procedure on
that machine.
- Python 3.12 or newer (available as ``python3`` in your PATH)
- `Podman <https://podman.io/docs/installation>`_ (with ``podman compose``)
- Git
- At least **64 GB** of free disk space
- Recommended: 8 CPU cores, 16 GB RAM

The virtual machine must be provisioned with at least 64 GB of disk space.
I've compiled with 8 virtual CPUs and 16GB of RAM, though it may be possible
with less than that.
Quick Start
-----------

Setting up Ren'Py Build requires some Linux knowledge to complete.
Clone this repository and the Ren'Py source::

I recommend dedicating a user to Ren'Py Build. In this example, I name the
user ``rb``, with a home directory of ``/home/rb``. Once that's done, you
will want to modify your computer so that user can use the ``sudo`` command
without a password. It's important that the username you chose does not have
a space in it.
git clone https://github.com/renpy/renpy-build
cd renpy-build
git clone https://github.com/renpy/renpy

That means first manually sudo-ing to root with the ``sudo -s`` command and
your user's password. Run the ``visudo`` command, and add the following line
to the bottom of the file:
Download the required third-party archives listed in `tars/README.rst
<tars/README.rst>`_ and place them in the ``tars/`` directory.

rb ALL = (ALL) NOPASSWD : ALL
Then build::

Be sure to leave a blank line after it, then save the file with ctrl+X, and
use ``exit`` to get back to the non-root user. Note that this will allow
anyone who can log in as rb to become the superuser of this system.
./run.sh build

On Windows, use ``run.ps1`` instead of ``run.sh``::

Preparing
---------
.\run.ps1 build

To get ready to build, log in as the rb user, and then run the following
commands to instal git and clone renpy-build::
This will:

sudo apt install git
git clone https://github.com/renpy/renpy-build
1. Build the container image (first run only).
2. Run the full Ren'Py build inside the container.

Subsequent builds reuse the image and only rebuild components that changed.

Build Options
~~~~~~~~~~~~~

Options are passed before the ``build`` command::

./run.sh --platform linux --arch x86_64 build

``--platform <name>``
The platform to build for. One of ``linux``, ``windows``, ``mac``,
``android``, ``ios``, or ``web``.

Change into the renpy-build directory, and run prepare.sh::
``--arch <name>``
The architecture to build for. See the table below.

cd ~/renpy-build
./prepare.sh
``--nostrip``
Do not strip binaries.

**This will globally change your system.** It will install packages from Ubuntu
and LLVM repositories, and the UV website. Please make sure you're comfortable with
this change before continuing.
``--experimental``
Include experimental platforms.

This will first install all the packages required to build Ren'Py, and
then it will clone Ren'Py. It will also create a python
virtual environment with the tools in it. If this completes successfully,
you are ready to build.
``--stop <task>``
Stop after the specified task.

Finally, a number of files need to be downloaded from third parties. These
are listed in `tars/README.rst <tars/README.rst>`_ .
Supported Platform / Architecture Combinations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Building
---------
.. list-table::
:header-rows: 1

You'll need to be in the renpy-build directory to build. If you're not, run::
* - Platform
- Architecture
* - linux
- x86_64
* - linux
- aarch64
* - windows
- x86_64
* - mac
- x86_64
* - mac
- arm64
* - android
- x86_64
* - android
- arm64_v8a
* - android
- armeabi_v7a
* - ios
- arm64
* - ios
- sim-x86_64
* - ios
- sim-arm64
* - web
- wasm

cd ~/renpy-build
Cleaning
~~~~~~~~

It should then be possible to build using the command::
To perform a clean build, run::

./build.sh
./run.sh clean

The build command can take some options:
Advanced Usage
--------------

`--python <version>`
The python version to build. Only 3 is currently valid.
Container Management
~~~~~~~~~~~~~~~~~~~~

`--platform <name>`
The platform to build for. One of linux, windows, mac, android, ios, or web.
``--no-cache``
Rebuild the container image from scratch, ignoring the cache.

`--arch <name>`
The architecture to build for. The architectures vary by platform,
here is a copy of the table from build.py. ::
``--dry-run``
Print the ``podman`` command that would be executed without running it.

Running Commands in the Container
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Platform("linux", "x86_64", "3")
Platform("linux", "aarch64", "3")
You can run an arbitrary command (default: ``bash``) inside the build
container::

Platform("windows", "x86_64", "3")
./run.sh exec
./run.sh exec ls /build

Platform("mac", "x86_64", "3")
Platform("mac", "arm64", "3")
Exporting and Importing the Build Volume
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Platform("android", "x86_64", "3")
Platform("android", "arm64_v8a", "3")
Platform("android", "armeabi_v7a", "3")
The build state lives in a Podman volume. You can export it to a tarball
and re-import it later (e.g. to share a partial build)::

Platform("ios", "arm64", "3")
Platform("ios", "sim-x86_64", "3")
Platform("ios", "sim-arm64", "3")
./run.sh export renpy-build-tmp.tar
./run.sh import renpy-build-tmp.tar

Platform("web", "wasm", "3")
Developer Mode
~~~~~~~~~~~~~~

.. note::

A second build should be faster than the first, as it will only rebuild
Ren'Py and other components that are likely to frequently
change.
Dev mode is intended for people working on renpy-build itself, not for
building Ren'Py from source.

Updating
---------
Dev mode mounts the renpy-build source directories into the container so
that changes to build tasks and scripts are reflected without rebuilding
the image::

It's possible to change renpy/ to be a symlink to your own
clones of those projects after the prepare step is complete. Updating
renpy-build itself may require deleting the tmp/ directory and a complete
rebuild, though simple changes may not require that. You may also need to
run prepare.sh again.
./run.sh --dev --platform linux build

Note
----
Expand Down
6 changes: 6 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
services:
renpy-build:
build:
context: .
dockerfile: Dockerfile
image: localhost/renpy-build:${TAG:-latest}
53 changes: 47 additions & 6 deletions renpybuild/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,9 @@
import shutil
from pathlib import Path
import subprocess
import shutil

import jinja2

import renpybuild.run

from typing import Any


Expand Down Expand Up @@ -226,7 +223,9 @@ def set_names(self, kind : str, task : str, name : str):
else:
self.var("dlpa", "{{distlib}}/py{{ python }}-{{ platform }}-{{ arch }}")

renpybuild.run.build_environment(self)
from .run import build_environment

build_environment(self)

def expand(self, s : str, **kwargs) -> str:
"""
Expand Down Expand Up @@ -319,15 +318,20 @@ def run(self, command : str, verbose : bool=False, quiet : bool=False, **kwargs)
"""

command = self.expand(command, **kwargs)
renpybuild.run.run(command, self, verbose, quiet)

from .run import run

run(command, self, verbose, quiet)

def run_group(self):
"""
Creates a run_group. This is a context manager with a run method,
that allows multiple commands to be run in parallel.
"""

return renpybuild.run.RunGroup(self)
from .run import RunGroup

return RunGroup(self)

def clean(self, d : str="{{build}}"):
"""
Expand Down Expand Up @@ -473,3 +477,40 @@ def compile(self, src : str|Path):
flags = f'-b {flags}'

self.run(command, flags=flags, src=src)

def clone(
self,
url: str,
options: str = "",
*,
directory: str = "",
minimal: bool = True,
submodules: bool = False,
):
"""
Clones the repository at `url` into the current directory.

`options`
Options to pass to git clone.

`directory`
The directory to clone into.

`minimal`
If true, automatically applies options to minimize download size.

`submodules`
If true, also clones submodules recursively.
"""

url = self.expand(url)
options = self.expand(options)
directory = self.expand(directory)

if minimal:
options = f"--depth 1 --no-tags --single-branch --shallow-submodules {options}"

if submodules:
options = f"--recurse-submodules {options}"

self.run(f"git clone {options} {url} {directory}")
Loading