Repository navigation
SNOW-3923354 decouple pandas version from python connector and support pandas 3.x #4312
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sfc-gh-jzeng
wants to merge
25
commits into
main
Choose a base branch
from
jzeng/snow-3923354-pandas3x-support
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 24 commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
051402b
SNOW-3923354 decouple pandas version from python connector and import…
sfc-gh-jzeng 115ef72
SNOW-3923354 preserve Series name on ColumnEmulator copies under pand…
sfc-gh-jzeng de8567e
SNOW-3923354 keep sort keys at object dtype so NULL survives as None
sfc-gh-jzeng e4bda40
SNOW-3923354 keep SQL NULL as None across pandas-3 dtype re-inference
sfc-gh-jzeng 2438471
SNOW-3923354 fix MERGE-INSERT default NULLs under pandas 3 Copy-on-Write
sfc-gh-jzeng 3c4e30f
SNOW-3923354 derive the generated-import expectation from read_sql.__…
sfc-gh-jzeng e577c41
SNOW-3923354 fix pandas-3 positional accessors in vectorized UDF tests
sfc-gh-jzeng 06f3cc3
SNOW-3923354 accept pandas-3 dtype labels in to_pandas cast expectations
sfc-gh-jzeng 975ddf6
SNOW-3923354 normalize timedelta64 to ns on the pandas write path
sfc-gh-jzeng f978573
SNOW-3923354 make the structured-type to_json expectation version-aware
sfc-gh-jzeng 85bdce9
SNOW-3923354 centralize pandas major version detection in utils
sfc-gh-jzeng 9b5a9bc
SNOW-3923354 update CHANGELOG with pandas 3 support notes
sfc-gh-jzeng 721986c
SNOW-3923354 rename the timedelta loop variable to avoid shadowing col
sfc-gh-jzeng 01c2c17
SNOW-3923354 run local-testing precommit on pandas 2 and 3
sfc-gh-jzeng 8bae44d
SNOW-3923354 normalize Arrow duration to ns on write_arrow
sfc-gh-jzeng 2897230
SNOW-3923354 keep NULL as None in mock initcap, greatest, and least
sfc-gh-jzeng 8d57a6d
SNOW-3923354 tighten the pandas 3 to_pandas changelog note
sfc-gh-jzeng 9c43219
SNOW-3923354 cover duration write call sites and initcap delimiters
sfc-gh-jzeng 666a198
SNOW-3923354 correct the timedelta changelog scope and trim the notes
sfc-gh-jzeng cf9a8eb
SNOW-3923354 derive pandas major version locally in tests
sfc-gh-jzeng a31458f
SNOW-3923354 drop the unused pandas_major_version helper
sfc-gh-jzeng c409453
SNOW-3923354 drop the redundant connector requirement from the pandas…
sfc-gh-jzeng 8aa19c0
SNOW-3923354 keep Arrow schema metadata when normalizing duration to ns
sfc-gh-jzeng 4c14a3c
SNOW-3923354 tighten the pandas 3 changelog entries
sfc-gh-jzeng 76bc2a9
SNOW-3923354 bound pyarrow in the pandas extra, mirroring the connector
sfc-gh-jzeng File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if I understand correctly, we add pyarrow here because we get rid of the [pandas] constraint?
If that is the case, do we need to align the pyarrow limit with connector? Like I remember they have a lower limit for pyarrow> xxx
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we previously relied on connector side which has pyarrow imported in its
[pandas]extra.Applied the same constraint mirroring with connector's side now.