Skip to content

New Metric: Upper Face Dynamics Deviation(FDD)#3238

Open
VijayVignesh1 wants to merge 46 commits intoLightning-AI:masterfrom
VijayVignesh1:feature/face_dynamic_deviation
Open

New Metric: Upper Face Dynamics Deviation(FDD)#3238
VijayVignesh1 wants to merge 46 commits intoLightning-AI:masterfrom
VijayVignesh1:feature/face_dynamic_deviation

Conversation

@VijayVignesh1
Copy link
Copy Markdown
Contributor

@VijayVignesh1 VijayVignesh1 commented Sep 2, 2025

What does this PR do?

Implements the Upper Face Dynamics Deviation (FDD) metric for 3D talking head evaluation.
Computes the deviation in motion magnitude of upper-face vertices between predicted and ground truth meshes.

Fixes #3097

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃


📚 Documentation preview 📚: https://torchmetrics--3238.org.readthedocs.build/en/3238/

@github-actions github-actions Bot added documentation Improvements or additions to documentation topic: MModal labels Sep 2, 2025
@VijayVignesh1 VijayVignesh1 marked this pull request as ready for review September 3, 2025 12:16
@rittik9 rittik9 force-pushed the feature/face_dynamic_deviation branch from f48ea3c to 2a74503 Compare September 4, 2025 16:25
@codecov
Copy link
Copy Markdown

codecov Bot commented Sep 4, 2025

Codecov Report

❌ Patch coverage is 36.00000% with 48 lines in your changes missing coverage. Please review.
✅ Project coverage is 36%. Comparing base (d184220) to head (6ef4bfd).

Additional details and impacted files
@@          Coverage Diff           @@
##           master   #3238   +/-   ##
======================================
- Coverage      37%     36%   -0%     
======================================
  Files         349     351    +2     
  Lines       19901   19973   +72     
======================================
+ Hits         7264    7288   +24     
- Misses      12637   12685   +48     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread docs/source/links.rst Outdated
@rittik9 rittik9 force-pushed the feature/face_dynamic_deviation branch from 45752a3 to 12cc909 Compare September 5, 2025 11:10
Comment thread src/torchmetrics/functional/multimodal/fdd.py Outdated
@VijayVignesh1
Copy link
Copy Markdown
Contributor Author

@Borda The unittest error seems to coming from a timeout error from azure's end. Unrelated to the code.

Comment thread src/torchmetrics/functional/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/functional/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/functional/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/functional/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/multimodal/fdd.py
Comment thread src/torchmetrics/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/multimodal/fdd.py Outdated
@VijayVignesh1
Copy link
Copy Markdown
Contributor Author

@Borda @rittik9 The build docs failure seems to be coming from a different broken link.

@Borda Borda requested a review from Copilot September 19, 2025 19:49
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds the Upper Face Dynamics Deviation (FDD) metric for evaluating 3D talking head upper-face motion, including functional and class APIs, documentation, and tests.

  • Introduces functional API upper_face_dynamics_deviation and class UpperFaceDynamicsDeviation
  • Wires exports, adds docs and plotting test coverage, and updates changelog

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 13 comments.

Show a summary per file
File Description
src/torchmetrics/functional/multimodal/fdd.py Adds the functional FDD computation and docstring
src/torchmetrics/multimodal/fdd.py Adds the metric class implementation and plotting support
src/torchmetrics/functional/multimodal/init.py Exposes the new functional
src/torchmetrics/multimodal/init.py Exposes the new metric class
src/torchmetrics/functional/init.py Re-exports the functional at top-level
tests/unittests/multimodal/test_fdd.py Adds unit tests for class, functional, errors, differentiability, and plotting
tests/unittests/utilities/test_plot.py Adds the metric to generic plot tests
docs/source/multimodal/fdd.rst Adds user-facing docs for the metric
docs/source/links.rst Adds reference link for FDD
CHANGELOG.md Notes the new metric addition

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread src/torchmetrics/functional/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/functional/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/functional/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/functional/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/multimodal/fdd.py Outdated
Comment thread src/torchmetrics/multimodal/fdd.py
Comment thread tests/unittests/multimodal/test_fdd.py Outdated
Comment thread docs/source/multimodal/fdd.rst Outdated
Comment thread docs/source/multimodal/fdd.rst Outdated
Comment thread CHANGELOG.md Outdated
@rittik9 rittik9 force-pushed the feature/face_dynamic_deviation branch from fdb763f to b3496d6 Compare September 29, 2025 19:55
@rittik9 rittik9 requested a review from Borda October 3, 2025 19:54
@mergify mergify Bot added the ready label Oct 3, 2025
@rittik9 rittik9 changed the title Adding Upper Face Dynamics Deviation metric New Metric Upper Face Dynamics Deviation(FDD) Oct 3, 2025
@rittik9 rittik9 force-pushed the feature/face_dynamic_deviation branch from e7d3421 to 44a4530 Compare January 30, 2026 10:43
@mergify mergify Bot added the ready label Jan 30, 2026
@mergify mergify Bot removed the ready label May 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation topic: MModal

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upper Face Dynamic Deviation (FDD) Metrics for 3D Talking Heads Evaluation

5 participants