Repository navigation
NO-SNOW: cast numeric prompt arg in AI complete test, drop expired date xfails - #4319
Merged
Merged
Conversation
… xfails test_dataframe_ai_complete_with_named_placeholders fails deterministically (26/26 daily precommit jobs, all clouds, through 5 pytest reruns) because AI_COMPLETE returns NULL for its prompt. Scope the xfail to that one test so the ~50 other AI tests in the file keep gating, and use strict=False so an XPASS surfaces once the underlying issue is resolved. The blanket date-based xfails in the module pytestmark and the doctest hook expired on 2026-06-04, so they were dead code masking nothing.
sfc-gh-bkogan
commented
Aug 17, 2026
sfc-gh-bkogan
left a comment
Collaborator
Author
There was a problem hiding this comment.
SHIP WITH NITS
Clean, well-scoped test-only change: expired blanket xfails removed with no dangling references, and the narrow xfail matches the file's existing convention. One tradeoff worth acknowledging inline about strict=False masking non-target failures.
Reproduced on a live deployment: PROMPT() stores an integer argument as a number, and the executor only accepts text/file arguments, so the row comes back as a NULL completion. Point the xfail at that instead of "infrastructure".
PROMPT() only accepts string arguments, so the integer rating column reached the server as a number and every row came back as a NULL completion. Cast it so the test exercises named placeholders instead of tripping over arg typing.
Docs allow NUMBER prompt args; the runtime currently rejects them. Make the comment match that rather than implying strings are the only supported type.
sfc-gh-bkogan
marked this pull request as ready for review
August 17, 2026 23:05
sfc-gh-bkogan
requested review from
sfc-gh-aling,
sfc-gh-jzeng and
sfc-gh-yixie
August 17, 2026 23:05
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4319 +/- ##
=======================================
Coverage 95.23% 95.24%
=======================================
Files 171 171
Lines 44720 44720
Branches 7676 7676
=======================================
+ Hits 42591 42593 +2
+ Misses 1341 1340 -1
+ Partials 788 787 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
sfc-gh-jzeng
approved these changes
Aug 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
test_dataframe_ai_complete_with_named_placeholdershas been failing in every daily precommit job (26/26, all clouds) since the blanket date-basedxfails insrc/conftest.pyandtests/integ/test_dataframe_ai.pyexpired on 2026-06-04. Deterministic, not a flake.Root cause: the test passes an integer
ratingcolumn intoPROMPT(). Official docs say NUMBER is allowed ("any type coercible to a string (for example, VARCHAR, NUMBER, etc.), or FILE"), but the runtime stores the number verbatim and rejects it withunsupported argument type. Withreturn_error_detailsoff, that surfaces as aNULLcompletion.Minimal repro:
Sibling
llama3.1-8btests pass because they only pass string columns.Changes
ratingtoStringTypeininput_columnsso this test covers named-placeholder rewriting instead of tripping over the server-side type bug.xfails (doctest hook insrc/conftest.py, module-level mark intest_dataframe_ai.py).Server-side follow-up for the AI SQL team: either cast string-coercible args in
PROMPT()as documented, or have the prompt parser stringify scalars. Separately, a type error collapsing to silentNULLis a bad failure mode.Test plan
test_dataframe_ai_complete_with_named_placeholdersgreen.dataframe_aitests and doctests now run unmasked; confirm no other latent failures.Verified the casted SQL against a live deployment: all three rows return non-empty completions. Pytest integ run is covered by CI (no local snowpark integ credentials here).