Skip to content

Fix required field default rendering and --use-default nullable types#3054

Merged
koxudaxi merged 9 commits into
koxudaxi:mainfrom
butvinm:fix/required-field-defaults
May 2, 2026
Merged

Fix required field default rendering and --use-default nullable types#3054
koxudaxi merged 9 commits into
koxudaxi:mainfrom
butvinm:fix/required-field-defaults

Conversation

@butvinm
Copy link
Copy Markdown
Contributor

@butvinm butvinm commented Mar 16, 2026

Fixes #3048

Two changes:

1. Consistent required field defaults (fixes the reported inconsistency)

__set_validate_default_on_fields now skips required fields, so model-ref fields no longer get defaults while scalar fields don't. All required fields behave the same — no defaults rendered without --use-default.

2. --use-default no longer makes fields nullable (breaking change)

Previously --use-default turned status: str | None = 'active' — making required fields optional and nullable. Now it produces status: str = 'active' — default is rendered but the type stays non-nullable. I consider the previous behavior a bug: the schema says the type is string, not string | null, and --use-default shouldn't change that.

Before (bug)

Using the schema from #3048, without --use-default:

class Order(BaseModel):
    order_id: str
    status: Status
    priority: Priority
    quantity: int
    note: str
    tags: list[str]
    metadata: dict[str, str]
    shipping_address: Address = Field(
        {'street': '123 Main St', 'city': 'Springfield'}, validate_default=True
    )
    all_addresses: list[Address] = Field(
        [{'street': '1 First St', 'city': 'Shelbyville'}], validate_default=True
    )
    addresses: dict[str, Address] = Field(
        {'home': {'street': '10 Oak Ave', 'city': 'Ogdenville'}}, validate_default=True
    )

Scalar required fields (status, quantity, etc.) have defaults dropped, but model-ref required fields (shipping_address, etc.) render defaults — inconsistent.

After (fix)

Without --use-default — consistent, all required fields have no defaults:

class Order(BaseModel):
    order_id: str
    status: Status
    priority: Priority
    quantity: int
    note: str
    tags: list[str]
    metadata: dict[str, str]
    shipping_address: Address
    all_addresses: list[Address]
    addresses: dict[str, Address]

With --use-default — all defaults rendered, types stay non-nullable:

class Order(BaseModel):
    order_id: str
    status: Status = 'pending'
    priority: Priority = 2
    quantity: int = 1
    note: str = 'your note here'
    tags: list[str] = ['new']
    metadata: dict[str, str] = {'source': 'web'}
    shipping_address: Address = Field(
        {'street': '123 Main St', 'city': 'Springfield'}, validate_default=True
    )
    all_addresses: list[Address] = Field(
        [{'street': '1 First St', 'city': 'Shelbyville'}], validate_default=True
    )
    addresses: dict[str, Address] = Field(
        {'home': {'street': '10 Oak Ave', 'city': 'Ogdenville'}}, validate_default=True
    )

Summary by CodeRabbit

  • New Features

    • Option to apply default values to fields while keeping them required and non-nullable.
  • Bug Fixes

    • Preserve default/default_factory for required fields when the option is enabled.
    • Avoid emitting placeholder-only representations for required fields when defaults are used.
  • Tests

    • Added E2E tests and updated expected fixtures covering required-with-default behavior across JSON Schema, OpenAPI, and GraphQL.

butvinm and others added 2 commits March 16, 2026 05:49
__set_validate_default_on_fields set validate_default=True on fields
with model references without checking field.required, causing required
model-ref fields to render defaults while required scalar fields didn't.

Skip required fields (unless use_default_with_required) so all required
fields consistently have no defaults rendered.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
--use-default previously skipped setting field.required=True, which made
fields non-required and therefore nullable (e.g. str | None = 'foo').

Now fields stay required=True with a new use_default_with_required flag
that allows defaults to render without changing the type. Produces
`status: str = 'foo'` instead of `status: str | None = 'foo'`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 16, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds use_default_with_required: bool to DataModelFieldBase and propagates it from GraphQL, JSON Schema, and OpenAPI parsers. Field-generation and parser logic now preserve default/default_factory and validate_default for required fields only when this flag is true; prior behavior that mutated required is removed.

Changes

use_default_with_required propagation and handling

Layer / File(s) Summary
Data Shape
src/datamodel_code_generator/model/base.py
Added use_default_with_required: bool = False to DataModelFieldBase.
Core Representation / Serialization
src/datamodel_code_generator/model/dataclass.py, src/datamodel_code_generator/model/msgspec.py, src/datamodel_code_generator/model/pydantic_base.py
Field stringification and has-field-assignment logic updated to conditionally include or omit default/default_factory and the "..."/Field(...) placeholder based on use_default_with_required.
Parser Base Behavior
src/datamodel_code_generator/parser/base.py
__set_validate_default_on_fields skips setting validate_default for required fields when use_default_with_required is false. __override_required_field marks copied overridden fields with use_default_with_required = True when appropriate.
Schema Parsers / Wiring
src/datamodel_code_generator/parser/jsonschema.py, src/datamodel_code_generator/parser/graphql.py, src/datamodel_code_generator/parser/openapi.py
Compute use_default_with_required from required, effective_has_default, and parser config (apply_default_values_for_required_fields); pass it into data model field constructors. Removed previous behavior that toggled required off when defaults existed. Updated method signatures/call sites accordingly.
Tests / Expected Fixtures
tests/main/jsonschema/test_main_jsonschema.py, tests/data/expected/main/**
Added E2E tests and updated/generated expected outputs reflecting required fields retaining defaults (nullability/type annotations changed where defaults apply; default/default_factory/validate_default preserved when flag true).

Sequence Diagram

sequenceDiagram
    participant Parser as Parser (JSONSchema/GraphQL/OpenAPI)
    participant Flag as FlagComputation
    participant Field as DataModelFieldBase
    participant Serializer as FieldSerializer
    participant Output as GeneratedCode

    Parser->>Flag: provide required, effective_has_default, config
    Flag-->>Parser: use_default_with_required = required && effective_has_default && config.apply_default_values_for_required_fields
    Parser->>Field: instantiate(name, required, default, use_default_with_required)
    Field->>Serializer: request string representation
    alt use_default_with_required == true
        Serializer->>Output: include default/default_factory and validate_default
    else
        Serializer->>Output: omit defaults for required fields (emit "..." or Field(...))
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested labels

breaking-change-analyzed, breaking-change

Suggested reviewers

  • ilovelinux

Poem

🐰
A tiny flag hopped into the stream,
Defaults stayed cozy — no fractured scheme.
Required kept steady, no None in sight,
Fields held their values, snug and right.
The generator twitched its nose in delight.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: fixing required field default rendering and addressing the --use-default nullable types issue.
Linked Issues check ✅ Passed The PR implements both objectives from issue #3048: ensures consistent default handling for required fields and prevents --use-default from making fields nullable.
Out of Scope Changes check ✅ Passed All changes are scoped to the two bug fixes: required field default consistency and preserving non-nullable types with --use-default.
Docstring Coverage ✅ Passed Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Mar 16, 2026

Merging this PR will degrade performance by 18.35%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 11 regressed benchmarks
⏩ 98 skipped benchmarks1

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime test_perf_stripe_style_pydantic_v2 1.8 s 2.1 s -16.97%
WallTime test_perf_large_models_pydantic_v2 3.2 s 4 s -18.35%
WallTime test_perf_deep_nested 5.4 s 6.5 s -16.46%
WallTime test_perf_complex_refs 1.9 s 2.3 s -17.17%
WallTime test_perf_graphql_style_pydantic_v2 741.7 ms 896.6 ms -17.27%
WallTime test_perf_aws_style_openapi_pydantic_v2 1.7 s 2.1 s -18.26%
WallTime test_perf_kubernetes_style_pydantic_v2 2.4 s 2.8 s -16.87%
WallTime test_perf_all_options_enabled 6 s 7 s -14.72%
WallTime test_perf_openapi_large 2.6 s 3.2 s -18.05%
WallTime test_perf_duplicate_names 938.9 ms 1,139.9 ms -17.63%
WallTime test_perf_multiple_files_input 3.3 s 3.9 s -15.45%

Comparing butvinm:fix/required-field-defaults (44d1b63) with main (ece1783)

Open in CodSpeed

Footnotes

  1. 98 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 16, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (ece1783) to head (44d1b63).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main     #3054   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           88        88           
  Lines        18306     18325   +19     
  Branches      2116      2117    +1     
=========================================
+ Hits         18306     18325   +19     
Flag Coverage Δ
unittests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

butvinm and others added 2 commits March 16, 2026 09:29
Cover the previously uncovered code paths:
- allOf with $ref + sub-schema required fields + --use-default
- allOf with outer required fields + --force-optional

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@butvinm butvinm force-pushed the fix/required-field-defaults branch from 3f51eb9 to bcef9e0 Compare March 16, 2026 08:54
@butvinm butvinm marked this pull request as ready for review March 16, 2026 10:05
@koxudaxi
Copy link
Copy Markdown
Owner

@butvinm

Thanks for the PR!
This is an issue that has been on the agenda for a while. It looks like the PR is trying to fix multiple problems at once, so I want to check the related issues for each of them before reviewing. Please bear with me. I woke up early today and I'm a bit tired, so I'll set aside time tomorrow to take a proper look.
As a project policy, I'd like to avoid breaking changes, so I'm hoping we can have some kind of migration period.

@koxudaxi
Copy link
Copy Markdown
Owner

@ilovelinux If you have time, could you share your thoughts on this PR, the related issues, and the approach? I feel like the PR needs to be split since it addresses two separate problems.

@ilovelinux ilovelinux self-requested a review March 19, 2026 22:28
@ilovelinux
Copy link
Copy Markdown
Collaborator

@koxudaxi I'll give a look ASAP 🙂

@koxudaxi
Copy link
Copy Markdown
Owner

@ilovelinux Thanks! I haven't gone through all the related issues yet and this looks like it could have a wide impact, so I want to take my time. I'll be back tonight, no rush at all.

Copy link
Copy Markdown
Collaborator

@ilovelinux ilovelinux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @koxudaxi, @butvinm. Sorry for the late answer. Last week has been tough.

Note

I splitted this comment in multiple paragraph. I though I wouldn't have much to say, but I was wrong. I hope they are easier to read.

About the changes

Note

@koxudaxi

If you have time, could you share your thoughts on this PR, the related issues, and the approach?

I admit reviewing changes hasn't been easy. I feel like we should reduce code complexity to increase code maintainability. However it kinda makes sense to me. I'll try to do a more in-depth review after this comment to double-check the logic.

I appreciate @butvinm work and agree with the fixes introduced, because:

  • The first change allow a coherent usage of default values.
  • The second change fix None type hint for required properties with default value defined (which can't be None because they are required).

IMHO both changes make sense and reflect the expected user experience.

I also found an useful description in the OpenAPI docs1 (emphasis mine):

The default value to use for substitution, which SHALL be sent if an alternate value is not supplied.

E.g. We have a Python client and a Python server which both uses datamodel-code-generator. The server shouldn't populate ANY property with default values, instead the client should use default values and send them explicitly.

More generally, this adhere to the DRY principle: one side populate the data, the other side validates the data. Double-validation and, more important, double-population of default values, is error-prone.

As now, it isn't possible to adhere to the DRY principle using models generated by datamodel-code-generator. This PR make default values coherent avoiding double-population, and removes wrong None type hint improving data validation, allowing to apply DRY principle.

About the multiple-responsibilities

@koxudaxi

It looks like the PR is trying to fix multiple problems at once, so I want to check the related issues for each of them before reviewing.

I agree the two changes solve two different problems. However:

  1. I'm not sure if they are also independent of each other.
    • If they are interdependent, splitting would make things more complex since one of them would depend on the other.
  2. They are both breaking changes about the same topic and I feel like they should be released together to reduce the end-user impact.

Despite single-responsibility PRs are cleaner and easier to review, I think we could avoid splitting this PR because the two changes are small and share the same surface impact (= default values).

About the breaking changes

I don't know how could a migration period for the users looks like for a change like this. I am not sure it would be useful because there's no way to avoid the breaking change. E.g.: changing the default formatter is a breaking change, but the migration period allow the users to acknowledge that and pin the old ones in the configuration file; this, instead, changes the way default values are rendered. There's no way to avoid that for the user and there's nothing the user can do to mitigate the breaking change. This looks more like an "incompatible version migration".

What if, instead, we communicate to the user what did change as soon as the user run the new version? A warning about breaking changes would:

  • Reduce the cognitive effort to keep track of the breaking changes.
  • Increases the chances the user understand and acknowledge what and why that happened.
  • Encourage the user to open an issue if something is broken because of the breaking change (e.g. a corner case that is no longer handled).

In this case, I think the users would appreciate this PR since it's a bugfix and I don't think many people were relying on either the wrong generated type hints or the default values only for model-ref fields. I think, instead, many other people may be having some hard time because of this.


I'm not approving this PR because I'm taking some time to think about the impact, what @koxudaxi said, and all I have written. In the meanwhile, I'd like to hear what do you all think! 🙂

Also, a hot pizza is waiting for me! 🍕 Can't ignore Italian priorities 😆

Footnotes

  1. https://swagger.io/specification/#server-variable-object

@butvinm
Copy link
Copy Markdown
Contributor Author

butvinm commented Mar 26, 2026

@ilovelinux thanks for the thorough review! I agree on all points - especially about not splitting and treating this as a bugfix rather than a breaking change.

Building on that, I think we could go even further. Since this PR already introduces breaking changes, we could address the broader nullable/required fields problem rather than patching it incrementally.

Required fields have their defaults dropped by default. The sole purpose of --use-default is to change this. But looking at the git history, this was never an intentional design decision - it was an artifact of template ordering (PR #99, March 2020). When a user reported it (issue #224), --use-default was added as a quick opt-in. The maintainer's own
truth table from issue #242 shows name: int = 100 (non-nullable with default) as the intended output - which is exactly what this PR now produces.

If there's no evidence of use cases that depend on dropping defaults from required fields, we could always render defaults - removing the special case entirely and deprecating --use-default. This would simplify both the API and the code.

I understand this is a bigger breaking change, so I'm fine keeping this PR as-is and exploring that direction separately.

@ilovelinux
Copy link
Copy Markdown
Collaborator

@butvinm

If there's no evidence of use cases that depend on dropping defaults from required fields, we could always render defaults - removing the special case entirely and deprecating --use-default. This would simplify both the API and the code.

Deprecating --use-default isn't something we want because there are some use-cases where a given component shall not have the responsibility of populating fields with default values. I mentioned that in my previous comment (DRY principle).

However, since I think the end user expects that default values are used to populate fields "by default", we may consider making --use-default opt-out (e.g. --no-use-default or --ignore-default). 👀

I understand this is a bigger breaking change, so I'm fine keeping this PR as-is and exploring that direction separately.

Yes. Feel free to open an issue about that so we can explore this proposal deeply without polluting this PR. 🙂

@butvinm
Copy link
Copy Markdown
Contributor Author

butvinm commented Mar 26, 2026

@ilovelinux agreed, will file a separate issue. Worth noting that currently --use-default doesn't cover this use case either — it only controls required fields. Non-required fields with non-None defaults always render them, with no way to opt out. A broader --strip-default flag (superset of --strip-default-none) might be the right solution for that.

@koxudaxi
Copy link
Copy Markdown
Owner

koxudaxi commented Apr 4, 2026

@butvinm @ilovelinux

Thanks for the detailed discussion.
I think this PR is currently touching on two different topics:

  1. fixing the inconsistency reported in Default values for required fields are sometimes rendered and sometimes not #3048
  2. discussing the broader design of default handling

For this PR, I would prefer to limit the scope to the first one.
In other words, I think we should focus on:

  • making required fields with defaults behave consistently
  • preventing --use-default from making required fields nullable

From the behavior/correctness point of view, I think these changes are closer to bug fixes.
At the same time, since they change generated output, I also agree that they may feel like breaking changes for users depending on the previous behavior.

So I think the safest way to handle this is:

  • treat this as a bug fix with compatibility impact
  • document the generated output changes clearly in the release notes / changelog

For the broader design questions around --use-default, such as:

  • whether it should remain opt-in
  • whether it should become opt-out in the future
  • whether we need a more general option like --strip-default

I think it would be better to open a separate issue and discuss them there.
That would make this PR easier to review and help us avoid mixing the immediate fix with a wider API / design discussion.

Does this direction sound good to both of you?
If we agree on this direction, I think the next steps should be:

  • @butvinm, please update the PR description to make the scoped goal explicit, and add a short note about the compatibility impact
  • @butvinm, please open a separate issue for the broader default-handling discussion if you still want to continue that proposal
  • after that, we can continue reviewing this PR only within the limited scope above

butvinm and others added 2 commits April 26, 2026 01:05
Three regressions where the new use_default_with_required flag wasn't
propagated to all the rendering sites that decide whether a field has
an assignment. Each produces broken output on this branch but the
correct output on baseline (92bdc27).

1. pydantic_base.py:205 — early-return for required+nullable fields
   with no other Field args returned Field(...) without checking
   use_default_with_required, dropping the user's default.
   Repro: --use-default --strict-nullable, required nullable str with
   default. Got: x: str | None = Field(...). Want: x: str | None = 'hello'.

2. dataclass.py:has_field_assignment — sort-key helper didn't account
   for use_default_with_required, so a required field with default null
   (use_default_with_required=True, field.field empty) was sorted as
   no-assignment alongside truly required fields. Schema order was then
   preserved, placing the defaulted field before the non-defaulted one.
   Repro: --use-default --output-model-type dataclasses.dataclass, required
   nullable a (default null) before required b. Got: a before b
   (TypeError on import). Want: b before a. Same fix covers
   pydantic_v2.dataclass since it shares the helper.

3. msgspec.py:_has_field_assignment — same class of bug. Helper checked
   neither bool(field.field) nor use_default_with_required, breaking
   ordering for both string-defaulted and null-defaulted required fields.
   Repro: --use-default --output-model-type msgspec.Struct on the existing
   allof_required_use_default.json fixture. Got: name='unnamed' before tag
   (TypeError on import). Want: tag before name='unnamed'.

Adds three regression tests in tests/main/jsonschema/test_main_jsonschema.py
covering the three repros above.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@butvinm
Copy link
Copy Markdown
Contributor Author

butvinm commented Apr 25, 2026

Hi! Sorry for disappearing — it was a tough month. I agree with limiting the scope of this PR to the two fixes, and moving the broader default-handling discussion to a separate issue.

Merged latest main, caught and fixed a few implementation bugs. PR is ready to merge whenever you'd like.

I'll open the follow-up issue for the broader default-handling discussion soon.

@koxudaxi koxudaxi merged commit 160e507 into koxudaxi:main May 2, 2026
42 of 43 checks passed
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 2, 2026

Breaking Change Analysis

Result: Breaking changes detected

Reasoning: The PR contains two breaking changes to generated code output: (1) --use-default previously set required=False, making fields nullable (e.g., str | None = 'active'); now fields stay required=True and non-nullable (str = 'active') — the PR author explicitly acknowledges this as a breaking change. (2) Without --use-default, required model-ref fields that previously rendered defaults with validate_default=True now consistently omit defaults like all other required fields — this is a bug fix but still changes generated output. Additionally, four built-in Jinja2 templates were updated with the new use_default_with_required field attribute, meaning custom templates that replicate the default-rendering logic would need updates to support the new behavior.

Content for Release Notes

Code Generation Changes

  • --use-default no longer makes required fields nullable - Previously, --use-default turned required fields into optional nullable fields (e.g., status: str | None = 'active'). Now required fields keep their original non-nullable type and just get the default value rendered (e.g., status: str = 'active'). Users whose downstream code depends on these fields being Optional/nullable will need to update. (Fix required field default rendering and --use-default nullable types #3054)
  • Required model-ref fields no longer render defaults without --use-default - Previously, required fields referencing models (e.g., shipping_address: Address) inconsistently rendered defaults with validate_default=True while scalar required fields did not. Now all required fields consistently omit defaults unless --use-default is passed. Users who relied on the previous behavior where model-ref required fields had defaults rendered will see those defaults removed. (Fix required field default rendering and --use-default nullable types #3054)

Custom Template Update Required

  • Built-in Jinja2 templates now use field.use_default_with_required - The built-in templates for BaseModel, dataclass, pydantic_v2/dataclass, and msgspec were updated to check field.use_default_with_required alongside field.required when deciding whether to render defaults. Custom templates that replicate the old default-rendering logic (e.g., {%- if not field.required %}) will still work but won't support the new --use-default behavior for required fields. To get the updated behavior, custom templates should change conditions like not field.required to (not field.required or field.use_default_with_required). (Fix required field default rendering and --use-default nullable types #3054)

This analysis was performed by Claude Code Action

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2026

🎉 Released in 0.57.0

This PR is now available in the latest release. See the release notes for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Default values for required fields are sometimes rendered and sometimes not

3 participants