From d1bbebb461eeae28338676db5f589efc03d7789b Mon Sep 17 00:00:00 2001 From: Jacob Garber Date: Sat, 20 Jun 2026 13:14:20 -0600 Subject: [PATCH] Unify contributing guides The Linux, Mac, and Windows specific guides are out of date. Combine them into the main guide so we don't duplicate any information and can keep everything in one place. Also: - update the pre-commit hook version to avoid a warning - update the gcloud install script to latest Google instructions --- .pre-commit-config.yaml | 2 +- CONTRIBUTING.md | 47 ++++++++++---- LINUX_SETUP.md | 140 ---------------------------------------- MACOS_SETUP.md | 77 ---------------------- README.md | 5 -- WINDOWS_SETUP.md | 90 -------------------------- install_gcloud.sh | 4 +- 7 files changed, 37 insertions(+), 328 deletions(-) delete mode 100644 LINUX_SETUP.md delete mode 100644 MACOS_SETUP.md delete mode 100644 WINDOWS_SETUP.md diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 34d8c53c1..e3357cafe 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,7 +1,7 @@ # https://pre-commit.com/ repos: - repo: https://github.com/pre-commit/pre-commit-hooks - rev: v4.5.0 + rev: v6.0.0 hooks: - id: trailing-whitespace - id: end-of-file-fixer diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d1289846e..3237dbd0b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,7 +15,35 @@ You'll need: - [pipx](https://pipx.pypa.io/) for installing Python tools - [git](https://git-scm.com/) for version control -Both of these can be installed using your distribution's package manager or [Homebrew](https://brew.sh/) on Mac. +Both of these can be installed using your distribution's package manager or [Homebrew](https://brew.sh/) on Mac. On Windows we recommend development on Linux using [WSL2](https://learn.microsoft.com/en-us/windows/wsl/install). For example, + +**Ubuntu, Debian, and Mint:** + +```bash +sudo apt install -y git pipx +pipx ensurepath +``` + +**Fedora:** + +```bash +sudo dnf install -y git pipx +pipx ensurepath +``` + +**Arch Linux:** + +```bash +sudo pacman -S git pipx +pipx ensurepath +``` + +**Mac** + +```bash +brew install git pipx +pipx ensurepath +``` ### Initial setup @@ -55,27 +83,20 @@ Both of these can be installed using your distribution's package manager or [Hom poetry install --with dev,test,docs ``` - This installs the runtime dependencies along with the `dev`, `test`, and `docs` - [dependency groups](https://python-poetry.org/docs/managing-dependencies/#dependency-groups). + This installs the runtime dependencies along with the `dev`, `test`, and `docs` [dependency groups](https://python-poetry.org/docs/managing-dependencies/#dependency-groups). If you only need to run tests, `poetry install --with test` is sufficient. - **Recommended**: Use `poetry run` to run commands inside the virtual environment: + You can use `poetry run` to run commands inside the virtual environment: ```bash poetry run pytest poetry run python script.py ``` - **Optional**: If you prefer an interactive shell session, install the shell plugin first: - - ```bash - poetry self add poetry-plugin-shell - ``` - - Then activate the environment with: + Alternatively, if you prefer an interactive shell session, activate the environment using: ```bash - poetry shell + eval $(poetry env activate) ``` After activation, commands run directly inside the virtual environment: @@ -147,7 +168,7 @@ Both of these can be installed using your distribution's package manager or [Hom poetry run pytest -v tests/integration ``` - Tests will run slowly the first time, as data required for testing will be read from GCS. Subsequent runs will be faster as data will be cached locally in the "gcs_cache" folder. + Tests will run slowly the first time, as data required for testing will be read from GCS. Subsequent runs will be faster as data will be cached locally in the `gcs_cache` folder. 6. **Run typechecking** diff --git a/LINUX_SETUP.md b/LINUX_SETUP.md deleted file mode 100644 index 4610a2d66..000000000 --- a/LINUX_SETUP.md +++ /dev/null @@ -1,140 +0,0 @@ -# Developer setup (Linux) - -## 1. Install Git - -Choose the command for your Linux distribution: - -**Ubuntu, Debian, and Mint:** - -```bash -sudo apt update -sudo apt install -y git -``` - -**Fedora:** - -```bash -sudo dnf install -y git -``` - -**Arch Linux:** - -```bash -sudo pacman -S sudo -sudo pacman -S git -sudo pacman -S openssh -``` - -If your Arch install does not have `sudo` configured yet, run the commands above as `root`, then configure `sudo` for your user. - -## 2. Fork and clone this repo - -After forking the repository on GitHub, clone your fork. - -Use SSH if your SSH keys are set up: - -```bash -git clone git@github.com:[YOUR_GITHUB_USERNAME]/malariagen-data-python.git -cd malariagen-data-python -``` - -Use HTTPS if you prefer, or if you do not have SSH keys configured (common on WSL): - -```bash -git clone https://github.com/[YOUR_GITHUB_USERNAME]/malariagen-data-python.git -cd malariagen-data-python -``` - -## 3. Install pipx - -Choose the command for your Linux distribution: - -**Ubuntu, Debian, and Mint:** - -```bash -sudo apt update -sudo apt install -y pipx -pipx ensurepath -``` - -**Fedora:** - -```bash -sudo dnf install -y pipx -pipx ensurepath -``` - -**Arch Linux:** - -```bash -sudo pacman -S python-pipx -pipx ensurepath -``` - -Close and reopen your terminal to apply PATH changes. -If you prefer to reload the shell in-place, run: - -```bash -exec bash -``` - -## 4. Install Poetry and Python 3.12 - -The package requires `>=3.10,<3.13`. We use Poetry's built-in installer to handle the Python version universally across all distributions. - -```bash -pipx install poetry -poetry python install 3.12 -``` - -## 5. Create development environment - -```bash -poetry env use 3.12 -poetry install --extras dev -``` - -## 6. Install pre-commit hooks - -```bash -pipx install pre-commit -pre-commit install -``` - -Run pre-commit checks manually: - -```bash -pre-commit run --all-files -``` - -## 7. Run tests - -Run fast unit tests using simulated data: - -```bash -poetry run pytest -v tests/anoph -``` - -## 8. Google Cloud authentication (for legacy tests) - -To run legacy tests which read data from GCS, you'll need to [request access to MalariaGEN data on GCS](https://malariagen.github.io/vector-data/vobs/vobs-data-access.html). - -Once access has been granted, [install the Google Cloud CLI](https://cloud.google.com/sdk/docs/install): - -```bash -./install_gcloud.sh -``` - -Then obtain application-default credentials: - -```bash -./google-cloud-sdk/bin/gcloud auth application-default login -``` - -Once authenticated, run legacy tests: - -```bash -poetry run pytest --ignore=tests/anoph -v tests -``` - -Tests will run slowly the first time, as data will be read from GCS and cached locally in the `gcs_cache` folder. diff --git a/MACOS_SETUP.md b/MACOS_SETUP.md deleted file mode 100644 index 067d60a7b..000000000 --- a/MACOS_SETUP.md +++ /dev/null @@ -1,77 +0,0 @@ -# Developer setup (macOS) - -The Linux setup guide is available in [LINUX_SETUP.md](LINUX_SETUP.md). If you are on macOS, follow these steps instead. - -## 1. Install Miniconda - -Download and install Miniconda for macOS from https://docs.conda.io/en/latest/miniconda.html. -Choose the Apple Silicon installer if you have an Apple Silicon Mac, or the Intel installer otherwise. You can check with: -```bash -uname -m -# arm64 = Apple Silicon, x86_64 = Intel -``` - -After installation, close and reopen your terminal for conda to be available. - -## 2. Create a conda environment - -The package requires Python `>=3.10, <3.13`. Python 3.13+ is not currently supported. -```bash -conda create -n malariagen python=3.11 -conda activate malariagen -``` - -## 3. Fork and clone this repo - -Fork the repository on GitHub, then clone your fork: -```bash -git clone git@github.com:[username]/malariagen-data-python.git -cd malariagen-data-python -pip install -e ".[dev]" -``` - -## 4. Install pre-commit hooks -```bash -pre-commit install -``` - -Run pre-commit checks manually: -```bash -pre-commit run --all-files -``` - -## 5. Run tests - -Run fast unit tests using simulated data: -```bash -pytest -v tests/anoph -``` - -## 6. Google Cloud authentication (for legacy tests) - -To run legacy tests which read data from GCS, you'll need to [request access to MalariaGEN data on GCS](https://malariagen.github.io/vector-data/vobs/vobs-data-access.html). - -Once access has been granted, install the Google Cloud CLI: -```bash -brew install google-cloud-sdk -``` - -Then authenticate: -```bash -gcloud auth application-default login -``` - -This opens a browser — log in with any Google account. - -Once authenticated, run legacy tests: -```bash -pytest --ignore=tests/anoph -v tests -``` - -Tests will run slowly the first time, as data will be read from GCS and cached locally in the `gcs_cache` folder. - -## 7. VS Code terminal integration - -To use the `code` command from the terminal: - -Open VS Code → `Cmd + Shift + P` → type `Shell Command: Install 'code' command in PATH` → press Enter. diff --git a/README.md b/README.md index e10834765..9f23a6ccf 100644 --- a/README.md +++ b/README.md @@ -46,11 +46,6 @@ for release notes. To get setup for development, see [this video if you prefer VS Code](https://youtu.be/zddl3n1DCFM), or [this older video if you prefer PyCharm](https://youtu.be/QniQi-Hoo9A). -For detailed setup instructions, see: -- [Linux setup guide](LINUX_SETUP.md) -- [macOS setup guide](MACOS_SETUP.md) -- [Windows setup guide](WINDOWS_SETUP.md) -- [Google Colab (TPU) setup guide](docs/source/colab_tpu_runtime.rst) Detailed instructions can be found in the [Contributors guide](https://github.com/malariagen/malariagen-data-python/blob/master/CONTRIBUTING.md). ## AI use policy and guidelines diff --git a/WINDOWS_SETUP.md b/WINDOWS_SETUP.md deleted file mode 100644 index dcd01757e..000000000 --- a/WINDOWS_SETUP.md +++ /dev/null @@ -1,90 +0,0 @@ -# Windows Setup Guide - -To get setup for development on Windows, see -[this video if you prefer VS Code](https://youtu.be/zddl3n1DCFM), -or [this older video if you prefer PyCharm](https://youtu.be/QniQi-Hoo9A), -and the instructions below. - -## 1. Fork and clone this repo -```bash -git clone https://github.com/[username]/malariagen-data-python.git -cd malariagen-data-python -``` - -## 2. Install Python - -Download and install Python 3.10 from the official website: -https://www.python.org/downloads/windows/ - -During installation, check the box that says Add Python to PATH -before clicking Install. - -Verify the installation worked: -```bash -python --version -``` - -## 3. Install pipx and poetry -```bash -python -m pip install --user pipx -python -m pipx ensurepath -pipx install poetry -``` - -After running ensurepath, close and reopen PowerShell before continuing. - -## 4. Create and activate development environment -```bash -poetry install -poetry shell -``` - -## 5. Install pre-commit hooks -```bash -pipx install pre-commit -pre-commit install -``` - -## 6. Add upstream remote and get latest code -```bash -git remote add upstream https://github.com/malariagen/malariagen-data-python -git pull upstream master -``` - -Note: On Windows the default branch is called master, not main. - -## 7. Verify everything works -```bash -python -c "import malariagen_data; print('Setup successful!')" -``` - -## Common Issues on Windows - -**poetry not found after install** - -Close and reopen PowerShell, then try again. - -**git not recognized** - -Install Git from https://git-scm.com/download/win -and restart PowerShell. - -**python not recognized** - -Reinstall Python and make sure to check -Add Python to PATH during installation. - -**fatal: not a git repository** - -Make sure you are inside the malariagen-data-python -folder before running any git commands. -```bash -cd malariagen-data-python -``` - -**error: pathspec main did not match** - -On Windows use master instead of main. -```bash -git checkout master -``` diff --git a/install_gcloud.sh b/install_gcloud.sh index fc114416a..9243bed37 100755 --- a/install_gcloud.sh +++ b/install_gcloud.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash set -xeuo pipefail -curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-469.0.0-linux-x86_64.tar.gz -tar -xf google-cloud-cli-469.0.0-linux-x86_64.tar.gz +curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-linux-x86_64.tar.gz +tar -xf google-cloud-cli-linux-x86_64.tar.gz ./google-cloud-sdk/install.sh --path-update=false --quiet --no-compile-python