Replacing VGGSfM tracker with VGGT tracker module#1035
Open
Replacing VGGSfM tracker with VGGT tracker module#1035
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request replaces the VGGSfM tracker with the VGGT tracker head module and updates image loading to support non-square images. The PR also introduces several algorithmic changes to bundle adjustment and translation averaging.
Changes:
- Replaced VGGSfM tracker with VGGT track head implementation using
vggsfm_utilsmodule - Added new
load_image_batch_vggt_loadermethod supporting both "crop" and "pad" modes for non-square images - Converted camera, depth, and tracking head computations to float32 precision
- Added new evaluation and visualization tools for COLMAP reconstructions
- Introduced nonlinear Sim3 trajectory alignment for cluster merging
- Modified bundle adjustment to use hard equality constraints instead of soft priors
- Changed translation averaging noise model dimension from 3 to 2
Reviewed changes
Copilot reviewed 19 out of 20 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| gtsfm/frontend/vggt.py | Replaced VGGSfM tracker with VGGT track head; removed deprecated coordinate conversion functions; added float32 conversions |
| gtsfm/loader/loader_base.py | Added new load_image_batch_vggt_loader method supporting non-square image preprocessing with crop/pad modes |
| gtsfm/cluster_optimizer/cluster_vggt.py | Updated to use new VGGT loading method; removed image/inference resolution parameters; added pre-BA result storage |
| gtsfm/visualization/visualize_tracks.py | New script for visualizing reprojection errors on COLMAP reconstructions |
| gtsfm/evaluation/compare_colmap_outputs_by_cluster.py | New script for comparing cluster reconstructions against COLMAP baseline |
| gtsfm/cluster_merging.py | Added nonlinear Sim3 merging support using TrajectoryAlignerSim3 |
| gtsfm/common/gtsfm_data.py | Added update_camera_pose method for modifying camera poses in-place |
| gtsfm/bundle/bundle_adjustment.py | Changed first camera prior from soft PriorFactorPose3 to hard NonlinearEqualityPose3 |
| gtsfm/averaging/translation/averaging_1dsfm.py | Changed noise model dimension from 3 to 2; commented out Huber robust loss |
| gtsfm/configs/vggt.yaml | Updated with new tracking parameters and removed resolution settings |
| gtsfm/configs/vggt_megaloc.yaml | New configuration file for VGGT with MegaLoc descriptor |
| gtsfm/scene_optimizer.py | Added support for nonlinear Sim3 alignment; changed metrics saving condition |
| gtsfm/runner.py | Increased default worker memory limit from 16GB to 32GB |
| gtsfm/utils/metrics.py | Added store_full_data parameter to metrics functions |
| gtsfm/evaluation/metrics.py | Added rounding to 4 decimal places for scalar metrics |
| gtsfm/evaluation/compare_colmap_outputs.py | Added camera center visualization and CSV export functionality |
| gtsfm/cluster_optimizer/cluster_optimizer_cacher.py | Added cache subdirectory support and VGGT output re-export |
| gtsfm/cluster_optimizer/cluster_optimizer_base.py | Added run_bundle_adjustment_on_leaf parameter |
| create_tracks_viz.sh | New shell script with example visualization commands |
| .gitignore | Added pattern to ignore result directories |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+56
to
+71
| aTi_measurements = _create_unary_measurements(parent_scene) | ||
| parent_camera_ids = set(parent_scene.get_valid_camera_indices()) | ||
| valid_child_scenes = [] | ||
|
|
||
| for i, child_scene in enumerate(children_scenes): | ||
| child_camera_ids = set(child_scene.get_valid_camera_indices()) | ||
| common_camera_ids = parent_camera_ids & child_camera_ids | ||
| if len(common_camera_ids) == 0: | ||
| logger.warning("Child scene %d has insufficient overlap with parent, skipping", i) | ||
| continue | ||
| valid_child_scenes.append(child_scene) | ||
|
|
||
| if len(valid_child_scenes) == 0: | ||
| return parent_scene | ||
|
|
||
| aTi_measurements = _create_unary_measurements(parent_scene) |
There was a problem hiding this comment.
Line 56 creates aTi_measurements which is then overwritten on line 71 with the exact same call. The first assignment is redundant and should be removed. This appears to be duplicate code.
hkhanuja
commented
Jan 30, 2026
resolving comments
Fixing dependencies
Minor filtering changes to frontend/vggt.py
Pinning GTSAM version
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR does 4 major things: