Fix: EA attribute specifying whether copy=False is ignored by Aniketsy · Pull Request #63130 · pandas-dev/pandas

Aniketsy · 2025-11-16T16:10:48Z

closes not necessarily for this PR, but should we add an EA attribute specifying whether copy=False is ignored? that way we could avoid overriding tests? #63040 (Replace xxxx with the GitHub issue number)
Tests added and passed if fixing a bug or adding a new feature

Please let me know if my approach or fix needs any improvements . I’m open to feedback and happy to make changes based on suggestions.
Thank you!

Aniketsy · 2025-11-20T14:01:57Z

@jbrockmendel Could you please take a look in this, If i'm not in wrong direction.

Aniketsy · 2025-11-28T15:49:36Z

@jbrockmendel Please review these changes, whenever you get a chance.

jbrockmendel · 2025-12-02T15:51:17Z

-            data.fillna(data_missing[1], copy=False)
-        tm.assert_extension_array_equal(data, data_missing)
+        if self._supports_fillna_copy_false:
+            # but with copy=False, this raises for EAs that respect the copy keyword


i think this comment should be for EAs that dont respect the copy keyword?

I've added a comment in the else block for the EAs that don't respect the copy keyword.

ok but isnt the comment here still wrong?

i thought about removing that comment earlier but wasn’t fully sure since it was written specifically for the if case. You're right, so I’ll remove it.

jbrockmendel · 2025-12-02T15:52:44Z

-        result = data.fillna(data_missing[1])
-        assert result[0] == data_missing[1]
+        res_copy = data.fillna(fill_value, copy=True)
+        tm.assert_extension_array_equal(res_copy, expected)


can you keep using data_missing instead of defining a new expected. i.e. minimize the diff

Aniketsy · 2025-12-03T09:32:06Z

@jbrockmendel I've updated the code as per your suggestion. Please let me know if this also needs a whatsnew entry.

Aniketsy · 2025-12-08T11:15:19Z

@jbrockmendel Is this PR ready now? The two ci-failures seem to be expected, since they’re caused by these. @pytest.mark.xfail(reason="copy keyword is missing")
@pytest.mark.xfail(reason="copy=False is not Implemented")
please let me know if you'd like me to enhance anything further.

Aniketsy · 2026-01-01T17:06:07Z

Hi @jbrockmendel , just a gentle reminder on this when you get a chance. Thanks!

jbrockmendel · 2026-03-18T16:19:38Z

+        res_copy = data.fillna(fill_value, copy=True)
+        tm.assert_extension_array_equal(
+            res_copy, data_missing.fillna(fill_value, copy=True)
+        )


why is this assertion added? seems meaningless

yes, this can be removed. thanks!

jbrockmendel · 2026-03-18T16:23:55Z

@@ -6,6 +6,8 @@


 class BaseMissingTests:
+    _supports_fillna_copy_false = True


should the name say "respect" instead of "support"?

in this not fully sure, but I’ve updated it as per your suggestion.

Aniketsy · 2026-03-19T08:40:19Z

I took a help of llm for fixing ci-fails, and updated with some changes.

jbrockmendel · 2026-03-19T23:43:19Z

        filled : IntervalArray with NA/NaN filled
        """
-        if copy is False:
-            raise NotImplementedError


IntervalArray.fillna always copies regardless of the copy keyword, so i think NotImplementedError is not needed removing this will pass without error, and behavior stays, same. I would like to know your thoughts on this.

raising is correct here.

jbrockmendel · 2026-03-19T23:43:46Z

+    def fillna(self, value=None, limit=None, copy: bool = True):
+        # We intentionally ignore copy and always perform a copy to ensure
+        # the original array is not modified.
        return super().fillna(value=value, limit=limit, copy=True)


pass copy=copy?

or raise/warn if the user passes copy=False

pass copy=copy?

i think we can go with this . I'll update

jbrockmendel · 2026-03-24T15:32:51Z

FAILED pandas/tests/extension/decimal/test_decimal.py::TestDecimalArray::test_fillna_frame - AssertionError: Did not see expected warning of class 'Pandas4Warning'

Aniketsy · 2026-03-28T20:43:09Z

FAILED pandas/tests/extension/decimal/test_decimal.py::TestDecimalArray::test_fillna_frame - AssertionError: Did not see expected warning of class 'Pandas4Warning'
FAILED pandas/tests/extension/decimal/test_decimal.py::TestDecimalArray::test_fillna_series - AssertionError: Did not see expected warning of class 'DeprecationWarning'
FAILED pandas/tests/extension/decimal/test_decimal.py::TestDecimalArray::test_fillna_limit_frame - AssertionError: Did not see expected warning of class 'DeprecationWarning'
FAILED pandas/tests/extension/decimal/test_decimal.py::TestDecimalArray::test_fillna_limit_series - AssertionError: Did not see expected warning of class 'DeprecationWarning'

@jbrockmendel as we have passed copy directly, i think we can remove these test, please let me know your thoughts on this.

jbrockmendel · 2026-04-03T01:14:13Z

+
+    def test_fillna_readonly(self, data_missing):
+        pa_dtype = data_missing.dtype.pyarrow_dtype
+        if pa.types.is_duration(pa_dtype):


its ugly but you could patch _respects_fillna_copy_false inside a context

jbrockmendel · 2026-04-03T01:15:16Z

        data = data_missing.copy()
        data._readonly = True

        # by default fillna(copy=True), then this works fine
-        result = data.fillna(data_missing[1])
-        assert result[0] == data_missing[1]


can you restore this line somewhere (to check that the NAs actually got filled)

tm.assert_extension_array_equal(res_no_copy, res_copy), i think this already check, i also checked by restoring and got this failure on testing

def test_fillna_readonly(self, data_missing): fill_value = data_missing[1] data = data_missing.copy() data._readonly = True # by default fillna(copy=True), then this works fine res_copy = data.fillna(fill_value, copy=True) > assert res_copy[0] == fill_value ^^^^^^^^^^^^^^^^^^^^^^^^^ E AssertionError pandas\tests\extension\base\missing.py:134: AssertionError

That suggests that the fillna isn't actually working. That would indicate a real bug.

i've used llm to debug and updated with some changes and restored as per your suggestion.
On running locally all tests were passing

jbrockmendel · 2026-04-09T22:34:24Z

Can you figure out the remaining test fialures

Aniketsy · 2026-04-10T06:20:13Z

sorry, i don't know if i should invest more time in this, as i've left with 2 PR's and two are closed, please feel free to close these. i don't want to create extra work for you to review . Thanks @jbrockmendel ☺️

jbrockmendel · 2026-04-18T17:07:12Z

closed by #65179

EA attribute specifying whether copy=False is ignored

43be6b7

Aniketsy marked this pull request as draft November 16, 2025 16:13

EA attribute specifying whether copy=False is ignored

251ff22

Aniketsy marked this pull request as ready for review November 19, 2025 09:58

Merge remote-tracking branch 'upstream/main' into fix-63040

0d8bae3

Aniketsy marked this pull request as draft November 20, 2025 11:37

EA attribute specifying whether copy=False is ignored

0df38db

Aniketsy force-pushed the fix-63040 branch from 39e004f to 0df38db Compare November 20, 2025 13:36

jbrockmendel reviewed Nov 21, 2025

View reviewed changes

Comment thread pandas/tests/extension/base/missing.py Outdated

Aniketsy added 4 commits November 22, 2025 10:15

EA attribute specifying whether copy=False is ignored

453b25f

EA attribute specifying whether copy=False is ignored

5421db1

EA attribute specifying whether copy=False is ignored

3f6333e

EA attribute specifying whether copy=False is ignored

964a6d2

Aniketsy marked this pull request as ready for review November 22, 2025 06:56

EA attribute specifying whether copy=False is ignored

1bf16a2

jbrockmendel reviewed Dec 2, 2025

View reviewed changes

Comment thread pandas/tests/extension/test_sparse.py

EA attribute specifying whether copy=False is ignored

871f164

Aniketsy added 3 commits December 3, 2025 15:45

EA attribute specifying whether copy=False is ignored

33f796f

EA attribute specifying whether copy=False is ignored

b7325fa

EA attribute specifying whether copy=False is ignored

cbf76eb

jbrockmendel added the Testing pandas testing functions or related to the test suite label Mar 6, 2026

jbrockmendel reviewed Mar 18, 2026

View reviewed changes

Aniketsy added 2 commits March 19, 2026 01:24

update from support -> respect

5a373d5

EA attribute specifying whether copy=False is ignored

4178f66

jbrockmendel reviewed Mar 19, 2026

View reviewed changes

Aniketsy commented Mar 26, 2026

View reviewed changes

Comment thread pandas/tests/extension/decimal/array.py

Aniketsy and others added 3 commits March 26, 2026 15:34

Update pandas/tests/extension/decimal/array.py

963bd84

EA attribute specifying whether copy=False is ignored

d0bc508

pass copy=copy

e3da0bc

jbrockmendel reviewed Mar 30, 2026

View reviewed changes

Comment thread pandas/tests/extension/decimal/array.py Outdated

Aniketsy added 2 commits March 31, 2026 14:05

Revert xfail-test and copy=copy

cdfd705

EA attribute specifying whether copy=False is ignored

9584cd1

jbrockmendel reviewed Apr 3, 2026

View reviewed changes

EA attribute specifying whether copy=False is ignored

7fedd5c

jbrockmendel closed this Apr 18, 2026

		@@ -6,6 +6,8 @@


		class BaseMissingTests:
		_supports_fillna_copy_false = True

Uh oh!

Conversation

Aniketsy commented Nov 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Aniketsy commented Nov 20, 2025

Uh oh!

Uh oh!

Aniketsy commented Nov 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Aniketsy commented Dec 3, 2025

Uh oh!

Aniketsy commented Dec 8, 2025

Uh oh!

Aniketsy commented Jan 1, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Aniketsy Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Aniketsy commented Mar 19, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jbrockmendel commented Mar 24, 2026

Uh oh!

Uh oh!

Aniketsy commented Mar 28, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jbrockmendel commented Apr 9, 2026

Uh oh!

Aniketsy commented Apr 10, 2026

Uh oh!

jbrockmendel commented Apr 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

Aniketsy commented Nov 16, 2025 •

edited

Loading

Aniketsy Mar 18, 2026 •

edited

Loading