Draft: Renew CodeKeeper around AI workflow native MetricReport interface

[ebihara99999]

Summary

Renew CodeKeeper around a metric-native MetricReport interface designed for recurring human and AI-assisted code quality review.

This issue tracks the design direction and implementation progress toward the 1.0 interface.

Background

CodeKeeper originally came from an engineering management need: observing the relationship between codebase health, team behavior, and business context over time.

When direct code ownership is difficult, continuous metric observation can help reveal how teams are working with the codebase and where quality risks may be accumulating.

That motivation still matters, but the practical workflow has changed. In the current development environment, it is more useful for CodeKeeper to produce structured metric reports that humans and AI agents can review periodically.

The renewed direction is therefore not to optimize first for long-term time-series analysis, but to make each report useful as input for recurring AI-assisted code quality review.

Motivation

CodeKeeper’s core value is keeping code metric values visible over time, independent of whether metric offenses are enforced strictly in day-to-day development.

The renewed interface should make those metric values easy to review periodically. A typical workflow is:

1. Run CodeKeeper periodically or before quality review work.
2. Generate a JSON metric report.
3. Ask a human or AI agent to identify hotspots, risks, and focused refactoring candidates.
4. Use metrics as review signals, not as hard pass/fail thresholds.

Why CodeKeeper in addition to RuboCop?

RuboCop is still important. It gives fast feedback, keeps code style consistent, and helps developers avoid many common problems during daily development.

A natural question is whether RuboCop Metrics cops can be reused as a metric collection interface. RuboCop can serialize offense results and summaries as JSON, but that output is structurally an offense report, not a metric inventory.

There are two important differences.

First, RuboCop Metrics output is threshold-gated. It reports offenses when values exceed configured limits such as Max. Values that stay under the threshold are not emitted as metric facts. For periodic code quality review, CodeKeeper needs the opposite default: collect metric values first, then let humans or agents decide which values deserve attention.

Second, RuboCop output is policy-filtered. Inline disable comments, Exclude, disabled cops, allowed method settings, and relaxed thresholds can suppress or hide values. This remains true even if RuboCop is configured aggressively to surface more metric offenses.

CodeKeeper’s role is to keep metric facts visible independently of offense thresholds and suppression policy. It complements RuboCop by producing structured metric reports that humans and AI agents can use for periodic quality review.

Design Decisions

• The central output object is MetricReport.
• measurements are the source of truth.
• summary is derived from measurements.
• summary.metrics.*.top_hotspots contains up to five measurements ordered by descending value, with deterministic ties by path, start line, and scope name.
• Metrics are measured at their natural scope.
• Scope names identify the owner: nested scopes carry their full namespace, and class << self with its methods resolves to the enclosing constant (class << Foo, Foo.bar) only where self statically is that constant, that is directly in a class or module body. Under method bodies and blocks the receiver stays self.
• File paths are location metadata, not the primary grouping model.
• File-level, directory-level, domain-level, or owner-level summaries can be derived downstream.
• JSON is the only output format. Tabular views such as CSV can be derived from measurements downstream.
• RuboCop is used as the metric calculation foundation where practical.
• RuboCop offense policy configuration is intentionally not applied.
• Time-series storage is not the primary interface, but can be built from repeated reports.

Metric Scopes

• abc_metric: method-like scopes
• cyclomatic_complexity: method-like scopes
• class_length: class/module-like scopes

Proposed JSON Interface

{
  "summary": {
    "metrics": {
      "abc_metric": {
        "count": 10,
        "max": 18.2,
        "top_hotspots": []
      }
    }
  },
  "measurements": []
}

Example measurement:

{
  "metric": "abc_metric",
  "scope_type": "method",
  "scope_name": "User#save!",
  "path": "app/models/user.rb",
  "start_line": 42,
  "end_line": 80,
  "value": 18.2
}

Implementation Plan

• Introduce metric-native Measurement objects.
• Introduce MetricReport as the central output object.
• Keep summary as derived data from measurements.
• Make default JSON output return the metric report schema.
• Remove CSV output so JSON is the only 1.0 output format.
• Align standard metric values with RuboCop calculation behavior.
• Ignore RuboCop offense configuration during measurement.
• Remove legacy metric engine behavior before 1.0.
• Remove legacy output format aliases before 1.0.
• Remove unnecessary direct dependencies.
• Define supported Ruby versions for 1.0.
• Update README with the AI-assisted review workflow.
• Update tests to compare CodeKeeper values with RuboCop metric behavior.

Non-goals

• Do not mirror .rubocop.yml policy configuration.
• Do not make file-level summary the primary data model.
• Do not make time-series storage the primary interface.
• Do not preserve legacy engine behavior in the 1.0 interface.
• Do not provide a CSV output format in the 1.0 interface.

Progress

Done

• Capture the renewal direction and proposed MetricReport interface in this issue.
• Clarify why RuboCop offense output is not a metric inventory.
• Create draft PR Renew metric report interface #39 for the 1.0 metric report interface work.
• Implement Measurement and MetricReport as the public output model.
• Align RuboCop-backed metric values with RuboCop calculation behavior.
• Remove legacy metric engine behavior and legacy output aliases.
• Document the AI-assisted review workflow in README.
• Remove unnecessary direct runtime dependencies.
• Define supported Ruby versions for 1.0.
• Remove CSV output support.

Remaining

• Finalize the draft PR after review.
• Update changelog and release notes during the release step, including the removal of CSV output and config.format.
• Decide whether additional custom-metric extractor/fact infrastructure belongs in this 1.0 increment or a follow-up issue.

Acceptance Criteria

• The public output model uses MetricReport.
• JSON output contains summary and measurements.
• JSON is the only supported output format.
• Measurements include metric name, scope type, scope name, path, line range, and value.
• Summary data is derived from measurements.
• top_hotspots returns a bounded, deterministic top-five list for each metric.
• RuboCop-compatible metric calculation behavior is used where available.
• RuboCop policy configuration does not suppress or alter measurements.
• README explains the AI-assisted review workflow.
• Legacy metric engine and legacy output aliases are removed before 1.0.
• Runtime dependencies are reviewed and unnecessary direct dependencies are removed.