If you are planning to develop GLASS, or want to use the latest commit of GLASS on your local machine, you might want to install it from the source. This installation is not recommended for users who want to use the stable version of GLASS. The page below describes how to build, test, and develop GLASS.
The developer installation of GLASS comes with several optional dependencies -
test: installs extra packages used in tests and the relevant testing framework/pluginsdocs: installs documentation related dependenciesexamples: installs libraries used in the examples and a few notebook related dependencies
These options can be used with uv in the following way
uv sync --extra examples --group docs --group testNote that the dev group is installed by default.
GLASS uses a set of prek hooks to format, lint, and prettify the codebase. The
hooks can be installed locally using -
prek installThis would run the checks every time a commit is created locally. The checks will only run on the files modified by that commit, but the checks can be triggered for all the files using -
prek run --all-filesIf you would like to skip the failing checks and push the code for further
discussion, use the --no-verify option with git commit. Alternatively, one
can run the linting workflow via nox:
nox -s lintGLASS is tested using pytest and pytest-doctestplus. pytest is responsible
for testing the code, whose configuration is available in
pyproject.toml.
pytest-doctestplus is responsible for testing the examples available in every
docstring, which prevents them from going stale. Additionally, GLASS also uses
pytest-cov (and Coveralls) to calculate/display the
coverage of these unit tests.
The tests can be executed using the test dependencies of GLASS in the
following way -
python -m pytest --cov --doctest-plusOne can specify a particular array backend for testing by setting the
ARRAY_BACKEND environment variable. The default array backend is NumPy. GLASS
can be tested with every supported array library available in the environment by
setting ARRAY_BACKEND to all. The testing framework only installs NumPy
automatically; hence, remaining array libraries should either be installed
manually or developers should use Nox.
# run tests using numpy
python -m pytest
ARRAY_BACKEND=numpy python -m pytest
# run tests using array_api_strict (should be installed manually)
ARRAY_BACKEND=array_api_strict python -m pytest
# run tests using jax (should be installed manually)
ARRAY_BACKEND=jax python -m pytest
# run tests using every supported array library available in the environment
ARRAY_BACKEND=all python -m pytestMoreover, one can test if a function is compatible with the array API standard
by using the xp fixture in tests. Further, the corresponding RNG Generator
can be accessed using the urng (a unified RNG interface) fixture. This will
parametrize the test to run on every array library specified through
ARRAY_BACKEND -
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from types import ModuleType
from glass._types import UnifiedGenerator
def test_something(xp: ModuleType, urng: UnifiedGenerator) -> None:
# use `xp.` to access the array library functionality
# use `urng.` to access the corresponding rng functionality
...GLASS supports JAX as an array backend and we would like to guarantee
auto-differentiability. To verify this, we have unit tests, which utilise
jax.test_util.check_grads,
defined in tests/gradients/.
These gradients tests can be run via nox:
uv run nox -s coverage_gradientsIn some of the gradients tests, a wrapper function has been defined in "force"
to auto-differentiability of the function being tested. This is needed as there
are a few aspects of a function which automatically cause it to fail the
jax.test_util.check_grads test:
- Backend mixing: the internals of
jax.test_util.check_gradsuses NumPy to apply a perturbation to the function inputs. If we have wrapped the function under test such that some of its inputs are static and have an associated array backend which have been fixed to JAX. When the perturbed inputs, which are now NumPy arrays, are passed intoarray_api_compat.array_namespacealongside the static inputs, we get the error -TypeError: Multiple namespaces for array inputs. The solution here is to wrap all inputs injnp.asarrayto ensure they have the same backend. - RNG calls: due to
jax.test_util.check_gradsmaking multiple calls to the function under test, if we define an inputrngoutside of these calls then the generated random number sequence will be different for each which breaks the auto-differentiability. The solution here is to define a freshrnginstance within a function that wraps the function under test and pass this wrapping function intoarray_api_compat.array_namespace. - Exact input value of 0.0: If 0.0 is used as an element of an input array,
there is a high chance that the arithmetic operation will result in a returned
value of exactly 0.0. It is then unlikely that
jax.text_util.check_gradswill be able to recover this exact 0.0 value after perturbing the input. The solution is to use a value close to but not exactly 0.0.
GLASS's documentation is mainly written in the form of
docstrings and
reStructuredText.
The docstrings include the description, arguments, examples, return values, and
attributes of a class or a function, and the .rst files enable us to render
this documentation on GLASS's documentation website.
GLASS primarily uses Sphinx for
rendering documentation on its website. The configuration file (conf.py) for
sphinx can be found
under the docs folder.
The documentation is deployed on
https://readthedocs.io.
Ideally, with the addition of every new feature to GLASS, documentation should
be added using comments, docstrings, and .rst files.
The documentation is located in the docs folder of the main repository. This
documentation can be generated using the docs dependencies of GLASS in the
following way -
cd docs/
make clean
make htmlThe commands executed above will clean any existing documentation build and
create a new build under the docs/_build folder. You can view this build in
any browser by opening the index.html file.
To release a new version of GLASS, there should be a commit that includes the following changes:
- The changes since the last release are added to the changelog
under a section titled
[yyyy.mm] (DD Mon YYYY). A new link to the changeset is added at the bottom of the file. - The release notes are updated with the new
version. The release notes should translate the changelog entries into prose
that can be understood by non-developer users of the code. If there are
breaking changes, a release note should explain what the changes mean for
existing code. There exists a script that helps generate an initial
draft of the release notes from the changelog
entries. We recommend running something like
bash .github/release_notes.sh | tee -a docs/manual/releases.rstto generate the draft and then editing it to make it more user-friendly.
Important
Make sure to test the release workflow by running the workflow_dispatch
trigger for the desired commit for the
Release workflow.
This test workflow will do a deployment to TestPyPI.
Once these changes are merged into the main branch, a new release with title
glass yyyy.mm should be created in the GitHub repository. The description of
the release should be a copy of its release note.
Creating the release will automatically start the build process that uploads Python packages for the new version to PyPI.
If any GLASS extension packages depend on the new release, new versions of these packages should be produced as soon as the new release is published to PyPI.
GLASS follows CalVer. There is no difference between releases that increment the year and releases that increment the month; in particular, releases that increment the month may introduce breaking changes.
The current version number is automatically inferred from the last release (i.e., git tag), subsequent unreleased commits, and local changes, if any.
The target is to have a new GLASS release once there are significant changes to the code's functionality.
GLASS supports running various critical commands using
nox to make them less intimidating for new
developers. All of these commands (or sessions in the language of nox) -
lint, tests, coverage, coverage_regression, coverage_gradients,
doctests, examples, docs, build, version, and regression_tests - are
defined in noxfile.py.
nox can be installed via uv using -
uv sync --only-devThe default sessions (lint and tests) can be executed using -
uv run noxA particular session (for example tests) can be run with nox on all
supported Python versions using -
uv run nox -s testsOnly tests, coverage and the doctests session run on all supported Python
versions by default.
To specify a particular Python version (for example 3.14), use the following
syntax -
uv run nox -s tests-3.14One can specify a particular array backend for testing by setting the
ARRAY_BACKEND environment variable. The default array backend is NumPy. GLASS
can be tested with every supported array library by setting ARRAY_BACKEND to
all.
# run tests using numpy
uv run nox -s tests-3.14
ARRAY_BACKEND=numpy uv run nox -s tests-3.14
# run tests using array_api_strict
ARRAY_BACKEND=array_api_strict uv run nox -s tests-3.14
# run tests using jax
ARRAY_BACKEND=jax uv run nox -s tests-3.14
# run tests using every supported array library
ARRAY_BACKEND=all uv run nox -s tests-3.14The following command can be used to deploy the docs on localhost -
uv run nox -s docs -- serveThe nox environments created for each type of session on the first run is
saved under .nox/ and reused by default.
To ensure that GLASS remains performant over time, a set of regression tests are provided in the regression folder. These regression tests can be run through nox. To compare the performance of two different revisions of GLASS.
uv run nox -s regression-tests -- <initial-state-revision> <revision-to-compare>To filter the tests to be ran, one can pass pytest arguments via the cli after
the required revision arguments. For example, if you wished to run only
regression tests from the test_fields.py file you could run one of the
following commands:
# Regression tests
uv run nox -s regression-tests -- <initial-state-revision> \
<revision-to-compare> -k test_fieldsTip
Regression tests should do minimal assertions on what is returned to make sure that the function is working as expected. The main goal of regression tests is to measure the performance of the function, and not to test its correctness. In the event that we testing for specific values the regression workflow may fail due to mismatches in values. See 01e6e4c for an example.
pytest-benchmark comes with a cProfile integration making it easy to profile
benchmark tests. To run the profiler, pass the additional flag
--benchmark-cprofile to pytest. To ensure the output is readable, you'll
likely want to limit the number of regression tests you profile by filtering
with -k <test_name>. See the
pytest-benchmark docs
for more details.
The regression tests are run in GitHub Actions for every pull request, which
updates relevant files, to ensure that new changes do not degrade the
performances of GLASS. There may be some small cases where the regression tests
are not automatically triggered but we believe it would be worth running them.
For example, if we update some dependency versions this will likely not trigger
the workflow. In this scenario, one can add the label run-regression-tests
which will then trigger the workflow.
This workflow uses the nox command detailed above comparing the head of the
current pull request to the main branch. The regression tests are first run
over the main branch, followed by the head of the pull request.
Note
There are two types of regression tests in GLASS: stable and unstable.
These are marked using the @pytest.mark.stable and @pytest.mark.unstable
decorators respectively. The stable tests are expected to have minimal
variance in their results across different runs, and are therefore measured in
percentage change during regression tests. The unstable tests may have
higher variance, and are therefore measured in absolute time change during
regression tests.
If there is any statistically significant regression in performance, the
workflow fails with the error
pytest_benchmark.session.PerformanceRegression: Performance has regressed. One
must scroll through the logs to find lines similar to the following to find out
which test(s) caused the regression:
--------------------------------------------------------------------------------
Performance has regressed:
test_displacement[numpy] (0001_8b154d8) - Field 'mean' has failed
PercentageRegressionCheck: 36.729528797 > 5.000000000
--------------------------------------------------------------------------------This indicates that the regression test test_displacement using the NumPy
array backend has regressed by approximately 36.73% compared to the previous
version. Effort has been made to reduce the number of false positives in these
regression tests, however some still occur. It is worth checking if the given
function (i.e. glass.displacement) has actually been hit by changes in the
given pull request (whether directly or indirectly). If not, the regression can
be ignored and merged. Sometimes this can be fixed by manually re-running the
workflow. If the regression is genuine, the workflow should repeatedly fail.
Regressions can be unavoidable sometimes, but every effort should be made to fix
them before merging the pull request.
Warning
A pull request that introduces a change such as the creation of a new module
or a change to the name of an existing module may cause the regression
workflow to fail because the new module wouldn't exist in main. Extra
caution should be taken in this case. It is recommended to run the regression
tests manually and look at the table of values.
Every change to the repository should come out of an issue where the change is discussed.
Pull requests should always follow from the discussion in an existing issue. The only exception are minor, obvious changes such as fixing typos in the documentation.
The discussion in a pull request should only be about the low-level details of its implementation. All high-level, conceptual discussion belongs to the issue to which the pull request refers.
Pull requests to the main branch should have titles of the following form:
gh-<issue-number>: Subject lineThe body of the pull request should contain a description of the changes, and any relevant details or caveats of the implementation.
The pull request should not repeat or summarise the discussion of its associated
issue. Instead, it should link to the issue using git's so-called "trailers".
These are lines of the form key: value which are at the end of the pull
request description, separated from the message body by a blank line.
To generically refer to an issue without any further action, use Refs and one
or more GitHub issue numbers:
Refs: #12
Refs: #25, #65To indicate that the pull request shall close an open issue, use Closes and a
single GitHub issue number:
Closes: #17Changelog entries are collected using the following trailers, and later parsed into the changelog for the next release:
Added: Some new feature
Changed: Some change in existing functionality
Deprecated: Some soon-to-be removed feature
Removed: Some now removed feature
Fixed: Some bug fix
Security: Some vulnerability was fixed
You can use any of the other common git trailers. In particular, you can use
Cc to notify others of your pull request via their GitHub user names:
Cc: @octocatGLASS uses All Contributors to acknowledge
different types of contributions to the project. The contributor data is stored
in .all-contributorsrc as JSON and is rendered as Markdown in
docs/CONTRIBUTORS.md. The markdown file is further rendered in
GLASS' documentation.
We encourage people to minimize contribution types, keeping them restricted to
code (usually includes docs, infra, and other related activities),
ideas, research, and funding.
To add a new contributor to the JSON and Markdown file, comment
@all-contributors please add @<username> for <contribution1>,<contribution2>,...on a PR/issue (a relevant PR/issue if it is a code contribution). The comment will trigger an automated PR from the all-contributors bot. Contributors should be added one at a time to avoid conflicts; one should wait for any existing PR by the all-contributors bot to be merged/closed before triggering a new one.
To acknowledge a contributor without a GitHub account, add an entry to
.all-contributorsrc manually and regenerate docs/CONTRIBUTORS.md using
all-contributors CLI
all-contributors generateIt is also often useful to run all-contributors CLI for debugging or fixing
generated files locally.