You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "mengw15 (via GitHub)" <gi...@apache.org> on 2026/05/28 20:17:05 UTC
[PR] fix(workflow-operator): auto-rename empty CSV column headers [texera]
mengw15 opened a new pull request, #5282:
URL: https://github.com/apache/texera/pull/5282
### What changes were proposed in this PR?
A CSV with an empty column header (e.g. a trailing comma `id,name,,age`) produced an `Attribute` with an empty name. After #3295 every output port writes via `IcebergTableWriter → Parquet`, where Avro rejects empty names with `IllegalArgumentException: Empty name` at flush time — losing the operator port's entire result.
Rename blank header positions to `column-<index>` (the convention used by pandas, Spark, R, and DuckDB) in all three CSV scan operators: `CSVScanSourceOpDesc`, `ParallelCSVScanSourceOpDesc`, and `CSVOldScanSourceOpDesc`. The issue named the first two; the legacy `csvOld` variant is still registered in `LogicalOp` and had the same latent bug.
Note: `ParallelCSVScanSourceOpDesc` is currently commented out of `LogicalOp`'s operator registry (so it is not reachable from the UI), but it is fixed here for consistency and so the bug does not resurface if it is re-enabled for experiments.
### Any related issues, documentation, discussions?
Closes #5279.
### How was this PR tested?
Added three cases to `CSVScanSourceOpDescSpec` — one per operator — feeding a CSV with an empty third header (`id,name,,age`) and asserting the inferred schema is `["id", "name", "column-3", "age"]`. `WorkflowOperator` suite is green (13 passed); scalafmt clean.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-opus-4-7)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(workflow-operator): auto-rename empty CSV column headers [texera]
Posted by "mengw15 (via GitHub)" <gi...@apache.org>.
mengw15 merged PR #5282:
URL: https://github.com/apache/texera/pull/5282
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(workflow-operator): auto-rename empty CSV column headers [texera]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #5282:
URL: https://github.com/apache/texera/pull/5282#issuecomment-4567873084
## [Codecov](https://app.codecov.io/gh/apache/texera/pull/5282?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 49.11%. Comparing base ([`ca829ec`](https://app.codecov.io/gh/apache/texera/commit/ca829ec086ec61b11cce84463e66b855e2d46189?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`7f534a9`](https://app.codecov.io/gh/apache/texera/commit/7f534a91640e0e15502b014dc95d11053a029dde?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #5282 +/- ##
=========================================
Coverage 49.11% 49.11%
Complexity 2378 2378
=========================================
Files 1051 1051
Lines 40342 40342
Branches 4277 4277
=========================================
Hits 19812 19812
Misses 19373 19373
Partials 1157 1157
```
| [Flag](https://app.codecov.io/gh/apache/texera/pull/5282/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| [access-control-service](https://app.codecov.io/gh/apache/texera/pull/5282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `39.53% <ø> (ø)` | | |
| [agent-service](https://app.codecov.io/gh/apache/texera/pull/5282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.76% <ø> (ø)` | | Carriedforward from [ca829ec](https://app.codecov.io/gh/apache/texera/commit/ca829ec086ec61b11cce84463e66b855e2d46189?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [amber](https://app.codecov.io/gh/apache/texera/pull/5282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `51.57% <ø> (ø)` | | Carriedforward from [ca829ec](https://app.codecov.io/gh/apache/texera/commit/ca829ec086ec61b11cce84463e66b855e2d46189?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/5282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <ø> (ø)` | | Carriedforward from [ca829ec](https://app.codecov.io/gh/apache/texera/commit/ca829ec086ec61b11cce84463e66b855e2d46189?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [config-service](https://app.codecov.io/gh/apache/texera/pull/5282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <ø> (ø)` | | |
| [file-service](https://app.codecov.io/gh/apache/texera/pull/5282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.99% <ø> (ø)` | | Carriedforward from [ca829ec](https://app.codecov.io/gh/apache/texera/commit/ca829ec086ec61b11cce84463e66b855e2d46189?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [frontend](https://app.codecov.io/gh/apache/texera/pull/5282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `41.06% <ø> (ø)` | | Carriedforward from [ca829ec](https://app.codecov.io/gh/apache/texera/commit/ca829ec086ec61b11cce84463e66b855e2d46189?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [python](https://app.codecov.io/gh/apache/texera/pull/5282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.79% <ø> (ø)` | | Carriedforward from [ca829ec](https://app.codecov.io/gh/apache/texera/commit/ca829ec086ec61b11cce84463e66b855e2d46189?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/5282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.81% <ø> (ø)` | | Carriedforward from [ca829ec](https://app.codecov.io/gh/apache/texera/commit/ca829ec086ec61b11cce84463e66b855e2d46189?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/texera/pull/5282?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org