Skip to content

feat: Add CLEAR MOT tracking evaluation metrics (MOTA, MOTP) with combined TrackingMetrics class#2193

Draft
Preksha2 wants to merge 2 commits intoroboflow:developfrom
Preksha2:feat/mota-tracking-metric
Draft

feat: Add CLEAR MOT tracking evaluation metrics (MOTA, MOTP) with combined TrackingMetrics class#2193
Preksha2 wants to merge 2 commits intoroboflow:developfrom
Preksha2:feat/mota-tracking-metric

Conversation

@Preksha2
Copy link
Copy Markdown

@Preksha2 Preksha2 commented Mar 30, 2026

Before submitting
  • Self-reviewed the code
  • Updated documentation, follow Google-style
  • Added docs entry for autogeneration (if new functions/classes)
  • Added/updated tests
  • All tests pass locally

Description

Adds CLEAR MOT tracking evaluation metrics to supervision's metrics module, implementing the standard defined in Bernardin & Stiefelhagen, "Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics" (JIVP, 2008). Includes MOTA (tracking accuracy), MOTP (tracking precision), and a combined TrackingMetrics class that computes both in a single pass.

Type of Change

  • ✨ New feature (non-breaking change which adds functionality)

Motivation and Context

Supervision has detection metrics (mAP, F1) but no tracking metrics. MOTA and MOTP are the two most widely used multi-object tracking evaluation metrics, standard in benchmarks like MOTChallenge. Tracking metrics were mentioned as welcome contributions in the 0.25.0 release notes. This PR fills that gap.

Changes Made

  • Added MultiObjectTrackingAccuracy class (MOTA) measuring overall tracking quality via false positives, false negatives, and identity switches
  • Added MultiObjectTrackingPrecision class (MOTP) measuring average localization precision of matched detections
  • Added TrackingMetrics class computing both MOTA and MOTP in a single pass (single IoU matrix computation and matching per frame)
  • Added MOTAResult, MOTPResult, MOTMetricsResult dataclasses with full component breakdown
  • Implemented greedy IoU-based matching with configurable threshold
  • Implemented identity switch detection across frames via ground truth ID tracking
  • Added input validation for missing tracker_id
  • All classes follow the existing update() / compute() interface consistent with the metrics API

Testing

  • I have tested this code locally
  • I have added unit tests that prove my fix is effective or that my feature works
  • All new and existing tests pass

30 tests covering: perfect tracking, false positives, false negatives, identity switches, IoU thresholds, partial overlap localization, empty frames, reset behavior, input validation, and string representations.

MOTA = 1 - (FN + FP + IDSW) / GT
MOTP = sum(IoU of matched pairs) / num_matches

Usage:

import numpy as np
from supervision.metrics.mota import TrackingMetrics
from supervision.detection.core import Detections

tracker = TrackingMetrics(iou_threshold=0.5)

gt = Detections(
    xyxy=np.array([[10, 10, 50, 50], [100, 100, 150, 150]]),
    tracker_id=np.array([1, 2]),
)
pred = Detections(
    xyxy=np.array([[12, 12, 52, 52], [105, 105, 155, 155]]),
    tracker_id=np.array([1, 2]),
)
tracker.update(gt, pred)
result = tracker.compute()
print(f"MOTA: {result.mota:.3f}, MOTP: {result.motp:.3f}")

Additional Notes

Happy to add documentation entries for autogeneration and adjust the API style if the maintainers have preferences. This is my first contribution to supervision.

@Preksha2 Preksha2 requested a review from SkalskiP as a code owner March 30, 2026 22:03
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 30, 2026

CLA assistant check
All committers have signed the CLA.

@Preksha2 Preksha2 force-pushed the feat/mota-tracking-metric branch from e50d010 to 424c9a9 Compare March 30, 2026 22:46
@Borda Borda marked this pull request as draft March 30, 2026 23:49
@Borda
Copy link
Copy Markdown
Member

Borda commented Mar 31, 2026

@Preksha2 thank you for your proposal. However, I think supervision will deprecate the tracking section since we created it as a separate repo - https://github.com/roboflow/trackers Am I right @SkalskiP" 🦝

@Preksha2
Copy link
Copy Markdown
Author

Thanks for the feedback @Borda! That makes sense — I wasn't aware tracking was being separated into its own repo.

I'd be happy to move this contribution to https://github.com/roboflow/trackers instead if that's the better home for tracking metrics. The implementation (MOTA, MOTP, TrackingMetrics) follows the same update()/compute() pattern and should integrate cleanly there.

Alternatively, if there's another area in supervision where metrics contributions are welcome, I'm happy to adapt. Let me know how you'd like me to proceed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants