-
Notifications
You must be signed in to change notification settings - Fork 97
Add support for non-branch coverage goals in DynaMOSA #142
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Aditya-9215
wants to merge
14
commits into
se2p:main
Choose a base branch
from
Aditya-9215:feature/dynamosa-line-coverage
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
c3f4d79
Add support for non-branch coverage goals in DynaMOSA
Aditya-9215 4567cb0
Apply pre-commit fixes (formatting, EOF)
Aditya-9215 e116d58
Fix mypy issues in type_inference
Aditya-9215 c54978a
Final CI fixes (ruff, typing, formatting)
Aditya-9215 7d30d0f
Revert unrelated changes in type_inference.py
Aditya-9215 209ec1d
Add support for non-branch coverage goals in DynaMOSA
Aditya-9215 7346aaa
Fix OpenAI compatibility and ensure LLM/DynaMOSA tests pass
Aditya-9215 1566327
Remove unrelated changes from PR
Aditya-9215 91c3112
Add dynamic activation for non-branch goals and corresponding test
Aditya-9215 a1c3539
Remove unintended dependency changes
Aditya-9215 3213ab5
Fix typing for mixed goal handling in DynaMOSA
Aditya-9215 472406c
Apply final pre-commit fixes
Aditya-9215 3595600
Reset DynaMOSA to base before proper generalization
Aditya-9215 516c14d
Add line coverage support to DynaMOSA with integration and behavior t…
Aditya-9215 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| def foo(x: int) -> int: | ||
| if x > 0: | ||
| return x + 1 | ||
| elif x == 0: | ||
| return 0 | ||
| else: | ||
| return x - 1 | ||
|
|
||
|
|
||
| def bar(y: int) -> int: | ||
| if y % 2 == 0: | ||
| return y * 2 | ||
| return y + 3 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| # This file is part of Pynguin. | ||
| # | ||
| # SPDX-FileCopyrightText: 2019–2026 Pynguin Contributors | ||
| # | ||
| # SPDX-License-Identifier: MIT | ||
|
|
||
| """Integration-like test for DynaMOSA handling of non-branch fitness functions.""" | ||
|
|
||
| from pynguin.ga.algorithms.dynamosaalgorithm import _GoalsManager | ||
| from pynguin.utils.orderedset import OrderedSet | ||
|
|
||
|
|
||
| class DummyArchive: | ||
| """Minimal archive.""" | ||
|
|
||
| def __init__(self): | ||
| self.covered_goals = set() | ||
| self.uncovered_goals = set() | ||
|
|
||
| def update(self, solutions): | ||
| pass | ||
|
|
||
| def add_goals(self, goals): | ||
| self.uncovered_goals = set(goals) | ||
|
|
||
|
|
||
| class DummySubject: | ||
| """Minimal subject properties.""" | ||
|
|
||
| existing_predicates = {} | ||
| existing_code_objects = {} | ||
|
|
||
|
|
||
| class DummyLineGoal: | ||
| """Represents a non-branch goal.""" | ||
|
|
||
| pass | ||
|
|
||
|
|
||
| def test_dynamosa_handles_only_non_branch_goals(): | ||
| """Ensure DynaMOSA works when only non-branch goals are present.""" | ||
|
|
||
| archive = DummyArchive() | ||
| subject = DummySubject() | ||
|
|
||
| line_goal = DummyLineGoal() | ||
|
|
||
| goals = OrderedSet([line_goal]) | ||
|
|
||
| manager = _GoalsManager(goals, archive, subject) | ||
|
|
||
| # Should include the non-branch goal | ||
| assert line_goal in manager.current_goals |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| # This file is part of Pynguin. | ||
| # | ||
| # SPDX-FileCopyrightText: 2019–2026 Pynguin Contributors | ||
| # | ||
| # SPDX-License-Identifier: MIT | ||
|
|
||
| """Integration test: DynaMOSA with line coverage.""" | ||
|
|
||
| import pynguin.configuration as config | ||
| from pynguin.generator import run_pynguin | ||
|
|
||
|
|
||
| def test_dynamosa_with_line_coverage(tmp_path): | ||
| """Ensure DynaMOSA works with line coverage on real code.""" | ||
|
|
||
| config.configuration.module_name = "tests.fixtures.simple_line_target" | ||
| config.configuration.algorithm = config.Algorithm.DYNAMOSA | ||
|
|
||
| # ✅ ADD THIS | ||
| config.configuration.test_case_output.coverage_metrics = ["LINE", "BRANCH"] | ||
|
|
||
| config.configuration.statistics_output.statistics_backend = ( | ||
| config.StatisticsBackend.NONE | ||
| ) | ||
|
|
||
| config.configuration.test_case_output.output_path = tmp_path | ||
|
|
||
| # Run Pynguin | ||
| run_pynguin() | ||
|
|
||
| assert tmp_path.exists() | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| # This file is part of Pynguin. | ||
| # | ||
| # SPDX-FileCopyrightText: 2019–2026 Pynguin Contributors | ||
| # | ||
| # SPDX-License-Identifier: MIT | ||
| # | ||
| """Tests for non-branch goal handling in DynaMOSA.""" | ||
|
|
||
| from typing import ClassVar | ||
|
|
||
| from pynguin.ga.algorithms.dynamosaalgorithm import _GoalsManager # noqa: PLC2701 | ||
| from pynguin.utils.orderedset import OrderedSet | ||
|
|
||
|
|
||
| def test_non_branch_goals_added_after_branch_completion(): | ||
| """Ensure non-branch goals activate only after branch goals are covered.""" | ||
|
|
||
| class DummyGoal: | ||
| """Simple dummy goal.""" | ||
|
|
||
| class DummyArchive: | ||
| """Minimal archive mock.""" | ||
|
|
||
| def __init__(self) -> None: | ||
| """Initialize archive state.""" | ||
| self.covered_goals = set() | ||
| self.uncovered_goals = set() | ||
|
|
||
| def update(self, solutions) -> None: | ||
| """Mock update.""" | ||
| return | ||
|
|
||
| def add_goals(self, goals) -> None: | ||
| """Track uncovered goals.""" | ||
| self.uncovered_goals = set(goals) | ||
|
|
||
| class DummySubject: | ||
| """Minimal subject properties mock.""" | ||
|
|
||
| existing_predicates: ClassVar[dict] = {} | ||
| existing_code_objects: ClassVar[dict] = {} | ||
|
|
||
| non_branch_goal = DummyGoal() | ||
|
|
||
| manager = _GoalsManager( | ||
| OrderedSet([non_branch_goal]), | ||
| DummyArchive(), | ||
| DummySubject(), | ||
| ) | ||
|
|
||
| # Initially, non-branch goals should NOT be active | ||
| assert non_branch_goal not in manager.current_goals |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unit tests with heavy mocking are great to test some behaviour of your code in isolation. In this case it is tested, that initially "non-branch" goals are not active, which makes sense if it is intended as it is in this case.
However, this does not test other properties of the algorithm, such as what happens once all branch goals are covered.
Even if that is also covered with a unit test with heavy mocking, it is still not tested that the behaviour is the same for non-mocked stuff. In general, using a simple non-mocked example with a real archive, real goals and a real subject is preferrable here.
Even if all of that is added, I would still not be convinced that now DynaMOSA + LineCoverage works. This must be tested with an integration test.