Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 49 additions & 16 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,36 +9,69 @@ This document provides project-specific context, conventions, and workflows for
## Build and Development

### Makefile Variables
- `DEBUG=1`: Enable debug symbols and disable optimizations.
- `ASAN=1`: Enable AddressSanitizer for memory safety checks.

| Variable | Default | Effect |
| ---------------------- | ------- | ----------------------------------------------------------------------------- |
| `DEBUG=1` | off | Enable debug symbols, disable optimisations, enable tree teardown on shutdown |
| `ASAN=1` | off | Enable AddressSanitizer |
| `UBSAN=1` | off | Enable UndefinedBehaviourSanitizer |
| `FUSE_MAJOR_VERSION=2` | 3 | Build against FUSE 2 instead of FUSE 3 (Linux only) |

### Key Makefile Targets
- `make all`: Build the `mount-zip` binary and the man page.
- `make check-fast`: Run the fast subset of tests.
- `make check`: Run the full test suite.
- `make doc`: Regenerate the `mount-zip.1` man page from `README.md`.
- `make clean`: Remove build artifacts.

| Target | Description |
| ----------------- | ---------------------------------------------------- |
| `make all` | Build the `mount-zip` binary and the man page |
| `make check-fast` | Run the fast subset of tests |
| `make check` | Run the full test suite (includes slow tests) |
| `make doc` | Regenerate `mount-zip.1` from `README.md` via pandoc |
| `make clean` | Remove all build artefacts |
| `make debug` | Shorthand for `DEBUG=1 make all` |
| `make valgrind` | Run tests under Valgrind (Linux only) |

### macOS Environment

The `Makefile` detects macOS at parse time via `uname -s`, queries the
Homebrew prefix with `brew --prefix`, and overrides `PKG_CONFIG` to carry
pinned search paths for macFUSE (`/usr/local`), ICU, and libzip. `CPPFLAGS`
is extended with the Boost include directory. Both are exported so all
sub-makes inherit them.

```sh
make
make check-fast
```

## Documentation Pipeline

The `README.md` file serves as both the user guide and the source for the man page.
- **Generation**: `pandoc` converts `README.md` to `roff` format.
- **Formatting**: The `Makefile` uses `sed` post-processing on the `pandoc` output to ensure bulleted lists are rendered compactly (using `.PD 0`) in the man page.
- **Markdown requirement**: Bulleted lists in `README.md` should be preceded by a blank line for correct `pandoc` parsing.

* **Generation**: `pandoc` converts `README.md` to `roff` format.
* **Formatting**: The `Makefile` uses `sed` post-processing on the `pandoc` output to ensure bulleted lists are rendered compactly (using `.PD 0`) in the man page.
* **Markdown requirement**: Bulleted lists in `README.md` should be preceded by a blank line for correct `pandoc` parsing.

## Technical Standards

### Memory Safety

The project aims for full **ASAN compliance**. Always verify changes with `ASAN=1 make check-fast`.

### Resource Management (RAII)
- Use RAII guards for resource cleanup (e.g., `ScopedFile`, `Cleanup`).
- **Shutdown Performance**: Global teardown of the virtual tree is wrapped in `#ifndef NDEBUG`. It is only performed in debug builds to keep production shutdown nearly instant.

* Use RAII guards for resource cleanup (e.g., `ScopedFile`, `Cleanup`).
* **Shutdown Performance**: Global teardown of the virtual tree is wrapped in `#ifndef NDEBUG`. It is only performed in debug builds to keep production shutdown nearly instant.

### Portability
- The project is 32-bit compatible.
- **Year 2038**: Always build with `-D_TIME_BITS=64` (handled in `Makefile`) to ensure correct timestamp handling on 32-bit systems.

* The project is 32-bit compatible.
* **Year 2038**: Always build with `-D_TIME_BITS=64` (handled in `Makefile`) to ensure correct timestamp handling on 32-bit systems.
* **macOS**: The source guards Apple-specific differences with `#ifdef __APPLE__`. Notable divergences:
* No `memfd_create()` — `lib/reader.cc` uses a temp-file fallback.
* `typeof` must be undefined around the `fuse.h` include to satisfy `-pedantic`.
* `boost::hash_combine` requires an explicit `#include <boost/container_hash/hash.hpp>` because Apple Clang does not pull it in transitively.

## Testing
- **Main Runner**: `tests/blackbox/test.py`
- **Whitebox Tests**: C++ unit tests in `tests/whitebox/` using the [GoogleTest](https://github.com/google/googletest) framework.

* **Main Runner**: `tests/blackbox/test.py`
* **Whitebox Tests**: C++ unit tests in `tests/whitebox/` using the [GoogleTest](https://github.com/google/googletest) framework.
* **Unmounting**: The blackbox test runner detects the host OS and uses `umount -f` on macOS and `umount -l` on Linux.
32 changes: 25 additions & 7 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@ To build **mount-zip**, you need the following libraries:
* [libfuse >= 3.1](https://github.com/libfuse/libfuse)
* [libzip >= 1.9.1](https://libzip.org)

On Debian systems, you can get these libraries by installing the following
packages:
### Debian / Ubuntu

```sh
$ sudo apt install libboost-container-dev libicu-dev libfuse3-dev libzip-dev
Expand All @@ -32,8 +31,6 @@ To build **mount-zip**, you also need the following tools:
* [GoogleTest](https://github.com/google/googletest) (for unit tests)
* [Pandoc](https://pandoc.org) to generate the man page

On Debian systems, you can get these tools by installing the following packages:

```sh
$ sudo apt install g++ pkg-config make libgtest-dev pandoc
```
Expand All @@ -43,12 +40,29 @@ To test **mount-zip**, you also need the following tools:
* `umount`
* [Python >= 3.8](https://www.python.org)

On Debian systems, you can get these tools by installing the following packages:

```sh
$ sudo apt install mount python3
```

### macOS

macOS requires [macFUSE](https://osxfuse.github.io) instead of libfuse. Install
it from the official disk image (it installs a kernel extension and requires
approval in **System Settings → Privacy & Security**):

```sh
$ brew install --cask macfuse
```

Then install the remaining dependencies via [Homebrew](https://brew.sh):

```sh
$ brew install boost icu4c libzip googletest pandoc
```

The `Makefile` detects macOS automatically and locates all Homebrew-installed
libraries without any extra environment variables.

## Build **mount-zip**

```sh
Expand All @@ -61,7 +75,7 @@ $ make
$ DEBUG=1 make
```

### With FUSE 2
### With FUSE 2 (Linux only)

```sh
$ FUSE_MAJOR_VERSION=2 make
Expand All @@ -81,6 +95,10 @@ $ make check
$ make check-fast
```

> [!NOTE]
> On macOS the test runner requires `python3` (from Xcode CLT or Homebrew) and
> uses `umount -f` automatically instead of the Linux-only `umount -l`.

## Install **mount-zip**

```sh
Expand Down
27 changes: 23 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,23 @@ PROJECT_CXXFLAGS += -DFUSE_USE_VERSION=26
endif

DEPS += libzip icu-uc icu-i18n

# On macOS, icu4c is keg-only (Homebrew does not symlink it into the default
# search path because macOS ships its own libicucore). Override PKG_CONFIG to
# carry the pinned icu4c path inline so every pkg-config call in this make and
# all sub-makes resolves the correct version regardless of what the shell
# environment (e.g. pkgx) may have injected.

ifeq ($(shell uname -s),Darwin)
BREW_PREFIX := $(shell brew --prefix 2>/dev/null)
ifneq ($(BREW_PREFIX),)
PKG_CONFIG = env PKG_CONFIG_PATH="$(BREW_PREFIX)/opt/icu4c/lib/pkgconfig" pkg-config
CPPFLAGS += -I$(BREW_PREFIX)/opt/boost/include
export PKG_CONFIG
export CPPFLAGS
endif
endif

PROJECT_CXXFLAGS += $(shell $(PKG_CONFIG) --cflags $(DEPS))
PROJECT_LDFLAGS = -L$(OUT) -lmountzip $(shell $(PKG_CONFIG) --libs $(DEPS))

Expand Down Expand Up @@ -102,12 +119,14 @@ $(MAN): README.md
-e 's/^\.TP/.PD\n.TP/g' > $@

install: $(DEST)
$(INSTALL) -D "$(DEST)" "$(DESTDIR)$(BINDIR)/mount-zip"
$(INSTALL) -D -m 644 $(MAN) "$(DESTDIR)$(MANDIR)/$(MAN)"
mkdir -p "$(DESTDIR)$(BINDIR)" "$(DESTDIR)$(MANDIR)"
$(INSTALL) "$(DEST)" "$(DESTDIR)$(BINDIR)/mount-zip"
$(INSTALL) -m 644 $(MAN) "$(DESTDIR)$(MANDIR)/$(MAN)"

install-strip: $(DEST)
$(INSTALL) -D -s "$(DEST)" "$(DESTDIR)$(BINDIR)/mount-zip"
$(INSTALL) -D -m 644 $(MAN) "$(DESTDIR)$(MANDIR)/$(MAN)"
mkdir -p "$(DESTDIR)$(BINDIR)" "$(DESTDIR)$(MANDIR)"
$(INSTALL) -s "$(DEST)" "$(DESTDIR)$(BINDIR)/mount-zip"
$(INSTALL) -m 644 $(MAN) "$(DESTDIR)$(MANDIR)/$(MAN)"

uninstall:
rm -f "$(DESTDIR)$(BINDIR)/mount-zip" "$(DESTDIR)$(MANDIR)/$(MAN)"
Expand Down
1 change: 1 addition & 0 deletions lib/node.h
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
#include <unistd.h>
#include <zip.h>

#include <boost/container_hash/hash.hpp>
#include <boost/intrusive/slist.hpp>
#include <boost/intrusive/unordered_set.hpp>

Expand Down
12 changes: 8 additions & 4 deletions tests/blackbox/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import hashlib
import logging
import os
import platform
import pprint
import random
import stat
Expand All @@ -26,6 +27,9 @@
import tempfile
import time

# macOS umount lacks -l (lazy); use -f (force) instead.
_UMOUNT_FLAG = '-f' if platform.system() == 'Darwin' else '-l'

sys.setrecursionlimit(3000)

# Computes the MD5 hash of the given file.
Expand Down Expand Up @@ -153,7 +157,7 @@ def MountZipAndGetTree(zip_names, options=[], password='', use_md5=True):
return GetTree(mount_point, use_md5=use_md5), os.statvfs(mount_point)
finally:
logging.debug(f'Unmounting {zip_paths!r} from {mount_point!r}...')
subprocess.run(['umount', '-l', mount_point], check=True)
subprocess.run(['umount', _UMOUNT_FLAG, mount_point], check=True)
logging.debug(f'Unmounted {zip_paths!r} from {mount_point!r}')


Expand Down Expand Up @@ -1909,7 +1913,7 @@ def TestBigZip(options=[]):
os.close(fd)
finally:
logging.debug(f'Unmounting {zip_path!r} from {mount_point!r}...')
subprocess.run(['umount', '-l', mount_point], check=True)
subprocess.run(['umount', _UMOUNT_FLAG, mount_point], check=True)
logging.debug(f'Unmounted {zip_path!r} from {mount_point!r}')


Expand Down Expand Up @@ -1950,7 +1954,7 @@ def TestManyNodes():

finally:
logging.debug(f'Unmounting {zip_path!r} from {mount_point!r}...')
subprocess.run(['umount', '-l', mount_point], check=True)
subprocess.run(['umount', _UMOUNT_FLAG, mount_point], check=True)
logging.debug(f'Unmounted {zip_path!r} from {mount_point!r}')


Expand Down Expand Up @@ -1994,7 +1998,7 @@ def TestBigZipNoCache(options=['-o', 'nocache']):
os.close(fd)
finally:
logging.debug(f'Unmounting {zip_path!r} from {mount_point!r}...')
subprocess.run(['umount', '-l', mount_point], check=True)
subprocess.run(['umount', _UMOUNT_FLAG, mount_point], check=True)
logging.debug(f'Unmounted {zip_path!r} from {mount_point!r}')


Expand Down
2 changes: 2 additions & 0 deletions tests/whitebox/extra_field_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@
#include <gtest/gtest.h>

#include <sys/stat.h>
#ifndef __APPLE__
#include <sys/sysmacros.h>
#endif
#include <cstdint>

namespace {
Expand Down