Metric Depth Models Mistake Digital Zoom for Moving Closer
Digital zoom (crop the centre, resize back) changes the camera, not the scene. The focal length grows by the zoom factor s and the true depth stays the same. We use this as a free, exact test for metric depth models and find that most of them read a zoom as the camera moving closer.
We measure two numbers per image, with no depth labels needed:
- α, depth exponent: depth ∝ s−α (ideal 0)
- β, focal exponent: focal ∝ sβ (ideal 1, only for models that predict their camera)
- Depth Anything V2 has α = 0.95: almost all of the zoom is read as moving closer.
- Models that predict their own camera (Depth Pro, MoGe-2, UniDepthV2) do better (α ≈ 0.31–0.36), but their focal grows too little.
- Given the true camera, the same networks keep their depth (α ≈ 0), so the error is in the focal head.
- FocalUnshrink, a fix with no labels and no training, brings Depth Pro to α = 0.04.
- EFT+B, fine-tuning with a zoom self-consistency loss, brings DA-V2-S to α = 0.05 (KITTI) and 0.07 (NYUv2).
Zooms 1–3×, 800 images (200 each from KITTI, NYUv2, DIODE indoor and outdoor), median over images with a 95%
interval from a scene-level bootstrap (as in Table 1 of the paper). Per-model CSV summary:
outputs/tables/table1_pooled.csv.
| Model | Camera | α (↓, ideal 0) | β (ideal 1) |
|---|---|---|---|
| DA-V2-S | pixels only | 0.95 [0.83, 1.12] | – |
| DA-V2-B | pixels only | 0.85 [0.76, 0.94] | – |
| ZoeDepth | pixels only | 0.61 [0.53, 0.75] | – |
| MoGe-2-S | predicts | 0.36 [0.30, 0.40] | 0.56 |
| UniDepthV2-S | predicts | 0.34 [0.23, 0.51] | 0.45 |
| Depth Pro | predicts | 0.31 [0.20, 0.41] | 0.65 |
| Metric3D v2-S | takes the true camera | −0.07 [−0.11, −0.05] | – |
| Metric3D v2-S | takes the old (un-zoomed) camera | 0.93 [0.89, 0.95] | – |
git clone <this repository> && cd zoom-equivariance
python -m venv .venv && . .venv/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/cpu # or a CUDA build
pip install -r requirements.txt
# two models are installed from their own repositories
git clone --depth 1 https://github.com/lpiccinelli-eth/UniDepth.git third_party/UniDepth
pip install --no-deps -e third_party/UniDepth
git clone --depth 1 https://github.com/YvanYin/Metric3D.git third_party/Metric3Drequirements-lock.txt has the exact versions behind every result.
bash scripts/download.sh # KITTI depth selection (val), NYUv2 labelled set, DIODE val -> data/raw/Sources, licences and the exact image ids of every sample are in DATA.md and outputs/samples/.
SR-RAW (real optical zoom) is described in DATA.md too.
Zoom one image and compare the model's depth:
import sys; sys.path.insert(0, "src")
import numpy as np
from PIL import Image
import models, zoom
model = models.load("depthpro") # any name from src/models.py REGISTRY
img = np.array(Image.open("my_photo.jpg").convert("RGB")) # uint8 H x W x 3
h, w = img.shape[:2]
box = zoom.zoom_box(h, w, s=2.0) # centred crop for a 2x zoom
img2 = zoom.zoom_image(img, box) # crop + bicubic resize back to H x W
d1 = model.predict(img)["depth"]
d2 = model.predict(img2)["depth"]
d1_crop = zoom.resize_depth(zoom.crop(d1, box), (h, w))
print("depth ratio, ideal 1:", np.median(d2 / d1_crop)) # about 1/2 means "moved closer"# audit: alpha, beta and errors against ground truth for one model on several sets
python scripts/run_audit.py depthpro kitti nyu diode-in diode-out --n 200
# controls used in the paper
python scripts/run_audit.py depthpro kitti --n 200 --off-centre # zoom into the four quadrants
python scripts/run_audit.py depthpro kitti --n 200 --cues # context loss vs magnification
python scripts/run_audit.py depthpro kitti --n 200 --jpeg-zoom # JPEG before / after the zoom
python scripts/run_audit.py metric3d-s kitti --n 200 --stale-k # give the old camera after zooming
python scripts/run_regions.py dav2-s kitti nyu --n 200 # alpha on objects vs walls / floors / roads
python scripts/run_srraw.py dav2-s --root data/raw/srraw/test # real optical zoom (SR-RAW) vs digital zoomEach run writes a per-image CSV to outputs/results/. Model names: dav2-s, dav2-b, dav2-l, zoedepth,
depthpro, moge2-s/b/l, unidepth2-s/b/l, unik3d-l, metric3d-s/l, da3-nested, da3-metric-l,
geocalib, and the -k variants (given the true camera).
# EFT / EFT+B: fine-tune DA-V2-S with the zoom loss (+ blur term, --w-blur) on unlabeled images
python scripts/train_eft.py outdoor --images "data/raw/kitti_raw/*/*/image_02/data/*.png"
python scripts/train_eft.py indoor --nyu-train
# EFT-S: frozen network, one learned log-scale per image (cache the frozen outputs, then fit)
python scripts/eft_scale.py cache outdoor --images "data/raw/kitti_raw/*/*/image_02/data/*.png"
python scripts/eft_scale.py fit outdoor
# EFT-F: tune only Depth Pro's focal head
python scripts/train_eft_focal.py --kitti "data/raw/kitti_raw/*/*/image_02/data/*.png" --nyu-trainOnly RGB is read; depth labels of the training images are never used. Training settings and every checkpoint are listed in CHECKPOINTS.md. Checkpoints are not in git.
All result CSVs are already in outputs/results/. To rebuild every table and number:
python scripts/make_paper_numbers.pyNo number in the paper is typed by hand; all of them come from this script. The large runs were done on Kaggle:
scripts/kaggle_build.py packs src/ and the job list into a kernel (credentials only through the
KAGGLE_USERNAME / KAGGLE_KEY environment variables).
src/ zoom geometry, datasets, model wrappers, audit, analysis, EFT trainers
scripts/ command-line entry points (audit, regions, SR-RAW, training, paper numbers)
kaggle/ kernel templates for the large runs
notebooks/ analysis notebooks
tests/ unit tests (no model weights needed)
outputs/ result CSVs, tables, figures, training logs
Run the tests with pytest.
This work builds on the released code and weights of Depth Anything V2, Depth Anything 3, ZoeDepth, Depth Pro, MoGe-2, UniDepthV2, UniK3D, Metric3D v2, VGGT, GeoCalib and SegFormer, and on the KITTI, NYUv2, DIODE and SR-RAW datasets. We thank their authors.
A BibTeX entry will be added after the review.
Code: MIT (see LICENSE). Datasets and model weights keep their own licences.
