You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@texera.apache.org by GitBox <gi...@apache.org> on 2026/08/14 10:16:58 UTC
[GH] (texera/gh-readonly-queue/main/pr-7568-8e80187a614fd8b82622c44594c6076eb69b80fe): Workflow run "Required Checks" failed!
The GitHub Actions job "Required Checks" on texera.git/gh-readonly-queue/main/pr-7568-8e80187a614fd8b82622c44594c6076eb69b80fe has failed.
Run started by GitHub user aglinxinyuan (triggered by aglinxinyuan).
Head commit for run:
c37f6ab7c68f15e7a1aa283d0c29b84e11f1ed6d / Kary Zheng <15...@users.noreply.github.com>
feat(csv-scan): keep a numeric column's type when one of its cells is blank (#7568)
### What changes were proposed in this PR?
Schema inference and execution disagreed about what a blank cell is, and
the schema side was the one that lost information. Inference set
`nullValue("")` on its parser, so a blank read as an empty string:
`tryParseDouble("")` fails, `tryParseBoolean("")` fails, and
`inferField` lands on `tryParseString()`. One empty cell was enough to
type a whole numeric column as STRING. Execution builds its parser
without `nullValue`, so the same blank read as null there, which is what
`AttributeTypeUtils.parseField` is written to pass through.
The effect reaches well past the scan. Every downstream operator that
does arithmetic on such a column then receives strings and fails on rows
whose values are perfectly good numbers, not on the blank one. Hugging
Face Iris Logistic Regression on a three-row file fails at the first
row, where numpy is handed `array([['2.6', '0.75']], dtype='<U32')`.
Dropping the setting leaves both sides reading a blank as null, and
`tryParseDouble(null)` already answers DOUBLE, so the column keeps the
type its values give it.
One corner changes with it: a column that is blank in every sampled row
now infers as INTEGER rather than STRING. Its values are null either
way, so this renames the empty rather than reinterpreting anything.
### Any related issues, documentation, discussions?
Closes #7550
### How was this PR tested?
`CSVScanSourceOpDescSpec` gains a case that writes a two-column file
whose numeric column is blank on one row and asserts the inferred type
is DOUBLE. It fails on the previous behavior, 16 passed / 1 failed
before the change and 17 / 0 after. Since this touches inference every
scan goes through, the whole module was run as well: 2187 passed, 0
failed.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5)
Co-authored-by: Claude Opus 5 (1M context) <no...@anthropic.com>
Co-authored-by: Xinyuan Lin <xi...@uci.edu>
Report URL: https://github.com/apache/texera/actions/runs/31788789027
With regards,
GitHub Actions via GitBox