Skip to content

#1585 - feat(better_networking): add structured response evaluator#1590

Open
soumyarajbag wants to merge 1 commit intofoss42:mainfrom
soumyarajbag:feat/response-evaluator
Open

#1585 - feat(better_networking): add structured response evaluator#1590
soumyarajbag wants to merge 1 commit intofoss42:mainfrom
soumyarajbag:feat/response-evaluator

Conversation

@soumyarajbag
Copy link
Copy Markdown
Contributor

@soumyarajbag soumyarajbag commented Mar 30, 2026

PR Description

Closes #1585. Introduces a structured, pure-Dart response evaluation utility in the better_networking package.

The current codebase has no way to programmatically assert API response correctness — only visual inspection via the UI. This PR adds
evaluateResponse(), a pure function that takes an HttpResponseModel and a ResponseExpectation, and returns a typed List<EvalResult>
one entry per configured assertion.

Supported assertions:

  • statusCode — exact match against expected HTTP status
  • maxLatency — response time within a duration threshold
  • requiredJsonKeys — top-level JSON keys must be present in the body
  • bodyContains — response body must contain a given substring
  • contentType — Content-Type header must contain a given substring

Why this approach over a simple logger:

A void logger with side effects is untestable and throws away the result. Returning List<EvalResult> lets callers (UI, tests, future batch
eval) decide what to do with the outcome — display it, aggregate it, or fail a test.

final results = evaluateResponse(
  response,
  const ResponseExpectation(
    statusCode: 200,
    maxLatency: Duration(milliseconds: 500),
    requiredJsonKeys: ['id', 'name'],
    bodyContains: 'success',
    contentType: 'application/json',
  ),
);

// results → List<EvalResult> with .field, .passed, .expected, .actual, .message

This also serves as the foundation for the multimodal AI & Agent API Eval Framework (Idea #2), where batch evaluation requires running
per-response assertions across datasets.

Related Issues

Checklist

  • I have gone through the contributing guide
  • I have updated my branch and synced it with project main branch before making this PR
  • I am using the latest Flutter stable branch (run flutter upgrade and verify)
  • I have run the tests (flutter test) and all tests are passing

Added/updated tests?

  • Yes
    • test/utils/response_evaluator_test.dart — 25 tests covering all 5 evaluation fields independently, edge cases (null body, null headers,
      null latency, empty keys list, long body truncation), and combined multi-assertion scenarios including result ordering

OS on which you have developed and tested the feature?

  • Windows
  • macOS
  • Linux

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add basic evaluation logging for API responses

1 participant