Skip to content

fix(jans-fido2): measure user-adoption metrics against the right population - #14860

Draft
imran-ishaq wants to merge 1 commit into
mainfrom
jans-fido2-user-adoption-metrics-population
Draft

fix(jans-fido2): measure user-adoption metrics against the right population#14860
imran-ishaq wants to merge 1 commit into
mainfrom
jans-fido2-user-adoption-metrics-population

Conversation

@imran-ishaq

Copy link
Copy Markdown
Contributor

Prepare


Description

Target issue

closes #14830

Implementation Details

Fido2MetricsService.getUserAdoptionMetrics reported three figures that did not mean what their
names said:

  • adoptionRate fell as adoption succeeded. It was newUsers / uniqueUsers, where uniqueUsers
    counted only users active in the query window. Once everyone had enrolled and was only signing in,
    newUsers tended to zero and the rate reported near-zero adoption exactly when adoption was
    complete.
  • newUsers did not mean first registration. The filter was REGISTRATION + SUCCESS inside the
    window, with no check for prior registrations — an existing user enrolling a second passkey counted
    as new, and the number changed meaning with the date picker.
  • returningUsers was derived by subtraction (uniqueUsers - newUsers), so a user who both
    registered and authenticated in the same window was counted only as new, never as returning.

The fix. getUserAdoptionMetrics now issues a second, targeted query against the metrics store —
getUsersRegisteredBefore(startTime) — for users whose registration already succeeded before the
window began ("prior adopters"). Everything downstream is derived from that set directly rather than
from window-only activity:

  • newUsers = registrations succeeding in the window, minus prior adopters — first-ever success only.
  • returningUsers = users active this window who are already prior adopters — computed directly, not
    by subtracting newUsers from uniqueUsers.
  • adoptionRate = newUsers / (priorAdopters + newUsers) — new users against the cumulative
    population of everyone who has ever registered as of endTime, so it tracks growth instead of
    falling toward zero as sign-in-only activity comes to dominate. When nobody has ever registered, the
    rate is null rather than a misleading 0.0.

This is intentionally the self-contained option: no new dependency on a directory-wide user count, and
no response-shape drop of adoptionRate — see the issue's "needs a product decision" note for the two
alternatives considered. It is bounded by the metrics retention policy: a user whose only prior
registration entry has already been cleaned up by cleanupOldData is reported as new again. That
tradeoff is documented on getUsersRegisteredBefore's Javadoc.

Unrelated bug found while implementing this, filed separately as #14859: Fido2AnalyticsService. generateExecutiveSummary recomputes its own adoption rate from totalUniqueUsers/newUsers instead
of using this method's adoptionRate, and casts those Integer values to Long, throwing
ClassCastException. Not touched here — it's a different file with its own review, and the class is
currently unwired from any controller.


Test and Document the changes

  • Static code analysis has been run locally and issues have been fixed
  • Relevant unit and integration tests have been added/updated
  • Relevant documentation has been updated if any (i.e. user guides, installation and configuration guides, technical design docs etc)

TestsFido2MetricsServiceTest, 5 added (34 total in the file, all green):

  • a user who registered before the window and only signs in during it is returning, not new
  • a user with no prior registration is new the first time they register
  • enrolling a second passkey does not make an already-adopted user look new again
  • adoptionRate is against cumulative adopters, not window activity
  • adoptionRate is null, not 0.0, when nobody has ever registered

Please check the below before submitting your PR. The PR will not be merged if there are no commits that start with docs: to indicate documentation changes or if the below checklist is not selected.

  • I confirm that there is no impact on the docs due to the code changes in this PR.

…lation

Signed-off-by: imran <imranishaq7071@gmail.com>
@coderabbitai

coderabbitai Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mo-auto

mo-auto commented Aug 25, 2026

Copy link
Copy Markdown
Member

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@mo-auto mo-auto added comp-jans-fido2 Component affected by issue or PR kind-bug Issue or PR is a bug in existing functionality labels Aug 25, 2026
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed for 'Fido2 API'

Failed conditions
1 New Bugs (required ≤ 0)
B Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp-jans-fido2 Component affected by issue or PR kind-bug Issue or PR is a bug in existing functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(jans-fido2): user-adoption metrics measure the wrong population

2 participants