You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "eugenegujing (via GitHub)" <gi...@apache.org> on 2026/07/10 23:01:23 UTC

[PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

eugenegujing opened a new pull request, #6323:
URL: https://github.com/apache/texera/pull/6323

   ### What changes were proposed in this PR?
   
   CSV/JSONL scan sources infer each column's type from only the first `INFER_READ_LIMIT` (= 100) rows. When a later row held a value that did not parse as the inferred type, the exec caught the error, mapped the row to `null`/`None`, and filtered it out — **silently dropping the entire row** with no error, warning, skipped-row count, or log. Every downstream count, aggregate, and join then ran on a silently truncated dataset. The bug is the silence, not the rejection; no comparable engine (Spark, Polars, DuckDB, Arrow) silently drops whole rows.
   
   ```
   Before:  row fails to parse  ->  caught -> null/None  ->  filtered out   (silent data loss)
   After:   row fails to parse  ->  RuntimeException      ->  console error + run stops
   ```
   
   Changes:
   
   - Throw a `RuntimeException` (original exception kept as cause) instead of silently dropping an unparsable row, in `CSVScanSourceOpExec`, `JSONLScanSourceOpExec`, `ParallelCSVScanSourceOpExec`, and `CSVOldScanSourceOpExec`. The throw surfaces through `DataProcessor.handleExecutorException` as a console error and stops the run (fail-fast, matching the default of Spark FAILFAST / Polars / DuckDB / Arrow).
   - Add a shared `ScanRowParseError` helper that identifies the offending column by re-parsing each field and builds a message that leads with the essential facts, so it stays useful even when the console title line truncates, e.g.:
     > Row 150: value '55.5' in column 'age' cannot be read as INTEGER. Column types were inferred from the first 100 rows of the file, and this value does not match. Fix the value in the file, or clean the data before scanning.
   
     A generic fallback covers rows with no single identifiable column (e.g. a malformed JSON line).
   - Remove the now-dead `null` filter in CSV and csvOld; `ParallelCSV` keeps its legitimate `null`-skip paths (exhausted block, all-null/blank line) and its filter — only parse failures abort.
   
   Inference logic is unchanged (the 100-row sampling is untouched). Scala only — no UI change.
   
   **Behavior-change note for reviewers:** workflows that previously ran to completion while silently discarding malformed rows will now fail on the first such row. This is the intended fix (loud over silent) and matches the modern-engine default, but it is a behavior change — see the discussion on #6279. Sample photo is attached below.
   
   ### Any related issues, documentation, discussions?
   
   Closes #6279
   
   ### How was this PR tested?
   
   Scan-operator unit tests (updated to assert the thrown error and message; added a JSONL malformed-line fallback test):
   
   ```
   sbt "WorkflowOperator/testOnly \
     org.apache.texera.amber.operator.source.scan.csv.CSVScanSourceOpExecSpec \
     org.apache.texera.amber.operator.source.scan.json.JSONLScanSourceOpExecSpec \
     org.apache.texera.amber.operator.source.scan.csv.ParallelCSVScanSourceOpExecSpec \
     org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpExecSpec"
   # => Tests: succeeded 28, failed 0
   ```
   
   Full backend gate in one clean session (lint + format + whole suite):
   
   ```
   sbt clean "scalafixAll --check" scalafmtCheckAll test
   # scalafixAll --check: pass
   # scalafmtCheckAll:    pass
   # test => Total number of tests run: 1721; succeeded 1721, failed 0, aborted 0 (pending 2)
   ```
   
   Manual, in the Texera UI: scanned a CSV whose column is integer for the first 100 rows with a non-integer value (`55.5`) at data row 150. The CSV File Scan operator turned red and the console showed the message above; the run stopped at row 150 instead of silently emitting a short result.
   <img width="1342" height="838" alt="Screenshot 2026-07-10 at 3 48 20 PM" src="https://github.com/user-attachments/assets/22df6a04-e97d-4818-80f3-e054afa8e7d6" />
   
   ### Was this PR authored or co-authored using generative AI tooling?
   
   Co-authored by: Claude Code (Fable 5)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

Posted by "eugenegujing (via GitHub)" <gi...@apache.org>.
eugenegujing commented on PR #6323:
URL: https://github.com/apache/texera/pull/6323#issuecomment-5027980283

   Closing in favor of the skip + report approach agreed in #6324. I will open a new PR fixing this issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

Posted by "eugenegujing (via GitHub)" <gi...@apache.org>.
eugenegujing closed pull request #6323: fix(workflow-operator): fail loudly on unparsable scan rows
URL: https://github.com/apache/texera/pull/6323


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6323:
URL: https://github.com/apache/texera/pull/6323#issuecomment-4940240593

   <!-- texera-benchmarks-comment -->
   ## ⚠️ Benchmark changes need a look
   
   🟢 0 better · 🔴 8 worse · ⚪ 7 noise (<±5%) · 0 without baseline
   
   > Compared against main `3e08ebb` benchmarked on **this same runner**, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.
   
   [Dashboard](https://apache.github.io/texera/dev/bench/) · [Run](https://github.com/apache/texera/actions/runs/29129296518)
   
   | | config | throughput | MB/s | latency | max Δ latest / 7d |
   |---|---|---:|---:|---:|---:|
   | 🔴 | bs=10 sw=10 sl=64 | 376 | 0.229 | 26,891/31,412/31,412 us | 🔴 +17.0% / 🔴 +121.9% |
   | 🔴 | bs=100 sw=10 sl=64 | 796 | 0.486 | 127,291/142,499/142,499 us | 🔴 +6.0% / 🔴 +38.1% |
   | 🔴 | bs=1000 sw=10 sl=64 | 898 | 0.548 | 1,110,430/1,231,620/1,231,620 us | 🔴 +6.1% / 🔴 +24.2% |
   
   <details><summary>Baseline details</summary>
   
   Latest main `3e08ebb` from same runner
   
   | config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
   |---|---|---:|---:|---:|---:|---:|
   | bs=10 sw=10 sl=64 | throughput | 376 tuples/sec | 428 tuples/sec | 814.14 tuples/sec | -12.1% | -53.8% |
   | bs=10 sw=10 sl=64 | MB/s | 0.229 MB/s | 0.261 MB/s | 0.497 MB/s | -12.3% | -53.9% |
   | bs=10 sw=10 sl=64 | p50 | 26,891 us | 22,992 us | 12,120 us | +17.0% | +121.9% |
   | bs=10 sw=10 sl=64 | p95 | 31,412 us | 29,376 us | 14,792 us | +6.9% | +112.4% |
   | bs=10 sw=10 sl=64 | p99 | 31,412 us | 29,376 us | 18,232 us | +6.9% | +72.3% |
   | bs=100 sw=10 sl=64 | throughput | 796 tuples/sec | 831 tuples/sec | 1,050 tuples/sec | -4.2% | -24.2% |
   | bs=100 sw=10 sl=64 | MB/s | 0.486 MB/s | 0.507 MB/s | 0.641 MB/s | -4.1% | -24.2% |
   | bs=100 sw=10 sl=64 | p50 | 127,291 us | 120,061 us | 96,227 us | +6.0% | +32.3% |
   | bs=100 sw=10 sl=64 | p95 | 142,499 us | 142,140 us | 103,174 us | +0.3% | +38.1% |
   | bs=100 sw=10 sl=64 | p99 | 142,499 us | 142,140 us | 111,033 us | +0.3% | +28.3% |
   | bs=1000 sw=10 sl=64 | throughput | 898 tuples/sec | 917 tuples/sec | 1,072 tuples/sec | -2.1% | -16.3% |
   | bs=1000 sw=10 sl=64 | MB/s | 0.548 MB/s | 0.559 MB/s | 0.655 MB/s | -2.0% | -16.3% |
   | bs=1000 sw=10 sl=64 | p50 | 1,110,430 us | 1,089,107 us | 950,645 us | +2.0% | +16.8% |
   | bs=1000 sw=10 sl=64 | p95 | 1,231,620 us | 1,161,156 us | 991,936 us | +6.1% | +24.2% |
   | bs=1000 sw=10 sl=64 | p99 | 1,231,620 us | 1,161,156 us | 1,022,498 us | +6.1% | +20.5% |
   
   </details>
   
   <details><summary>Raw CSV</summary>
   
   ```csv
   config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
   0,10,10,64,20,532.55,200,128000,376,0.229,26890.50,31412.29,31412.29
   1,100,10,64,20,2511.38,2000,1280000,796,0.486,127291.10,142498.84,142498.84
   2,1000,10,64,20,22270.25,20000,12800000,898,0.548,1110429.60,1231619.69,1231619.69
   ```
   
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

Posted by "eugenegujing (via GitHub)" <gi...@apache.org>.
eugenegujing commented on PR #6323:
URL: https://github.com/apache/texera/pull/6323#issuecomment-5027976680

   Closing in favor of the skip + report approach agreed in #6324. I will open a new PR fixing this issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

Posted by "Ma77Ball (via GitHub)" <gi...@apache.org>.
Ma77Ball commented on code in PR #6323:
URL: https://github.com/apache/texera/pull/6323#discussion_r3607785826


##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/csv/CSVScanSourceOpExec.scala:
##########
@@ -70,10 +71,16 @@ class CSVScanSourceOpExec private[csv] (descString: String) extends SourceOperat
             ): _*
           )
         } catch {
-          case _: Throwable => null
+          case e: Throwable =>
+            throw ScanRowParseError.build(
+              row.toSeq,
+              schema,
+              desc.INFER_READ_LIMIT,

Review Comment:
   When `limit` is set below 100 and `offset` shifts the output window past that smaller sample, a parse failure will claim "first 100 rows", sending the user to the wrong spot in the file. I recommend passing the actual sample size instead:
   
   ```suggestion
                 desc.limit.getOrElse(desc.INFER_READ_LIMIT).min(desc.INFER_READ_LIMIT),
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

Posted by "eugenegujing (via GitHub)" <gi...@apache.org>.
eugenegujing commented on PR #6323:
URL: https://github.com/apache/texera/pull/6323#issuecomment-4940693957

   > Since it is a behavior change, can you open a discussion post for this?
   
   Yes I raised one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #6323:
URL: https://github.com/apache/texera/pull/6323#issuecomment-4940226673

   ## [Codecov](https://app.codecov.io/gh/apache/texera/pull/6323?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 60.64%. Comparing base ([`a6c85b3`](https://app.codecov.io/gh/apache/texera/commit/a6c85b3b193de317f73f9b567aa9ceb687bc022b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`c43f9a9`](https://app.codecov.io/gh/apache/texera/commit/c43f9a940ea7d2758f4f0457052e7a91d1422862?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 32 commits behind head on main.
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##               main    #6323      +/-   ##
   ============================================
   - Coverage     60.64%   60.64%   -0.01%     
   + Complexity     3368     3367       -1     
   ============================================
     Files          1133     1133              
     Lines         44141    44139       -2     
     Branches       4825     4825              
   ============================================
   - Hits          26771    26769       -2     
     Misses        15908    15908              
     Partials       1462     1462              
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/texera/pull/6323/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
   |---|---|---|---|
   | [access-control-service](https://app.codecov.io/gh/apache/texera/pull/6323/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.00% <ø> (ø)` | | |
   | [agent-service](https://app.codecov.io/gh/apache/texera/pull/6323/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <ø> (ø)` | | Carriedforward from [a6c85b3](https://app.codecov.io/gh/apache/texera/commit/a6c85b3b193de317f73f9b567aa9ceb687bc022b?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [amber](https://app.codecov.io/gh/apache/texera/pull/6323/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `66.12% <ø> (-0.02%)` | :arrow_down: | Carriedforward from [a6c85b3](https://app.codecov.io/gh/apache/texera/commit/a6c85b3b193de317f73f9b567aa9ceb687bc022b?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/6323/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <ø> (ø)` | | |
   | [config-service](https://app.codecov.io/gh/apache/texera/pull/6323/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `52.30% <ø> (ø)` | | |
   | [file-service](https://app.codecov.io/gh/apache/texera/pull/6323/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.97% <ø> (ø)` | | Carriedforward from [a6c85b3](https://app.codecov.io/gh/apache/texera/commit/a6c85b3b193de317f73f9b567aa9ceb687bc022b?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [frontend](https://app.codecov.io/gh/apache/texera/pull/6323/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `52.40% <ø> (ø)` | | Carriedforward from [a6c85b3](https://app.codecov.io/gh/apache/texera/commit/a6c85b3b193de317f73f9b567aa9ceb687bc022b?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [notebook-migration-service](https://app.codecov.io/gh/apache/texera/pull/6323/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.94% <ø> (+0.37%)` | :arrow_up: | |
   | [pyamber](https://app.codecov.io/gh/apache/texera/pull/6323/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `91.18% <ø> (ø)` | | Carriedforward from [a6c85b3](https://app.codecov.io/gh/apache/texera/commit/a6c85b3b193de317f73f9b567aa9ceb687bc022b?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/6323/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.14% <ø> (ø)` | | Carriedforward from [a6c85b3](https://app.codecov.io/gh/apache/texera/commit/a6c85b3b193de317f73f9b567aa9ceb687bc022b?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   *This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/texera/pull/6323?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan commented on PR #6323:
URL: https://github.com/apache/texera/pull/6323#issuecomment-4940352805

   Since it is a behavior change, can you open a discussion post for this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

Posted by "chenlica (via GitHub)" <gi...@apache.org>.
chenlica commented on PR #6323:
URL: https://github.com/apache/texera/pull/6323#issuecomment-4975527028

   > > Since it is a behavior change, can you open a discussion post for this?
   > 
   > Yes I have raised one.
   
   @eugenegujing Please include the URL of the discussion.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6323:
URL: https://github.com/apache/texera/pull/6323#issuecomment-4940208016

   <!-- texera-reviewer-suggestion -->
   ### Automated Reviewer Suggestions
   
   Based on the `git blame` history of the changed files, we recommend the following reviewers:
   
   - **Contributors with relevant context:** `@aglinxinyuan`, `@Ma77Ball`, `@Yicong-Huang`
     You can notify them by mentioning `@aglinxinyuan`, `@Ma77Ball`, `@Yicong-Huang` in a comment.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): fail loudly on unparsable scan rows [texera]

Posted by "eugenegujing (via GitHub)" <gi...@apache.org>.
eugenegujing commented on PR #6323:
URL: https://github.com/apache/texera/pull/6323#issuecomment-4976427156

   https://github.com/apache/texera/discussions/6324


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org