You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "PG1204 (via GitHub)" <gi...@apache.org> on 2026/08/01 17:50:18 UTC

[PR] fix(workflow-operator): validate base64 in HF binary-column fallback … [texera]

PG1204 opened a new pull request, #7243:
URL: https://github.com/apache/texera/pull/7243

   ### What changes were proposed in this PR?
   
   In the HuggingFace operator's `_read_binary_value`, the final fallback decoded a column value with `base64.b64decode(val)`. Without `validate=True`, `b64decode` silently ignores characters outside the base64 alphabet, so ordinary text "successfully" decodes into garbage bytes and the intended `val.encode("utf-8")` fallback rarely fires and a plain-text cell could be turned into garbage and sent as image/audio bytes.
   
   Adding `validate=True` makes `b64decode` reject non-base64 input (spaces, punctuation, newlines), so real text now correctly falls through to UTF-8. Only this fallback call changes; the `data:`-URL decodes are left as-is (genuine base64 that can legitimately contain whitespace).
   
   ### Any related issues, documentation, discussions?
   
   Closes #7200.
   
   ### How was this PR tested?
   
   `sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.huggingFace.* org.apache.texera.amber.util.PythonCodeRawInvalidTextSpec"`: passes (125 tests). Added a test asserting the fallback uses `validate=True`; `PythonCodeRawInvalidTextSpec` py-compiles the generated Python. scalafmt clean.
   
   ### Was this PR authored or co-authored using generative AI tooling?
   
   This PP was co-authored with Claude Opus 4.8 in compliance with ASF policy.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): validate base64 in HF binary-column fallback … [texera]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7243:
URL: https://github.com/apache/texera/pull/7243#issuecomment-5152643345

   <!-- texera-reviewer-suggestion -->
   ### Automated Reviewer Suggestions
   
   Based on the `git blame` history of the changed files, we recommend the following reviewers:
   
   - No candidates found from `git blame` history.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): validate base64 in HF binary-column fallback … [texera]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #7243:
URL: https://github.com/apache/texera/pull/7243#issuecomment-5152652711

   ## [Codecov](https://app.codecov.io/gh/apache/texera/pull/7243?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 82.63%. Comparing base ([`33cc5d1`](https://app.codecov.io/gh/apache/texera/commit/33cc5d16096aa05095f483ee8724e799c95d7898?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e8ee1fa`](https://app.codecov.io/gh/apache/texera/commit/e8ee1fa441504aea5b5faec0b9ad09474be4085d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@            Coverage Diff            @@
   ##               main    #7243   +/-   ##
   =========================================
     Coverage     82.63%   82.63%           
     Complexity     4079     4079           
   =========================================
     Files          1162     1162           
     Lines         46275    46275           
     Branches       5160     5160           
   =========================================
     Hits          38241    38241           
     Misses         6343     6343           
     Partials       1691     1691           
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/texera/pull/7243/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
   |---|---|---|---|
   | [access-control-service](https://app.codecov.io/gh/apache/texera/pull/7243/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.00% <ø> (ø)` | | Carriedforward from [33cc5d1](https://app.codecov.io/gh/apache/texera/commit/33cc5d16096aa05095f483ee8724e799c95d7898?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [agent-service](https://app.codecov.io/gh/apache/texera/pull/7243/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.65% <ø> (ø)` | | Carriedforward from [33cc5d1](https://app.codecov.io/gh/apache/texera/commit/33cc5d16096aa05095f483ee8724e799c95d7898?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [amber](https://app.codecov.io/gh/apache/texera/pull/7243/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `79.31% <ø> (ø)` | | Carriedforward from [33cc5d1](https://app.codecov.io/gh/apache/texera/commit/33cc5d16096aa05095f483ee8724e799c95d7898?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/7243/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.60% <ø> (ø)` | | Carriedforward from [33cc5d1](https://app.codecov.io/gh/apache/texera/commit/33cc5d16096aa05095f483ee8724e799c95d7898?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [config-service](https://app.codecov.io/gh/apache/texera/pull/7243/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `65.97% <ø> (ø)` | | |
   | [file-service](https://app.codecov.io/gh/apache/texera/pull/7243/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `66.80% <ø> (ø)` | | Carriedforward from [33cc5d1](https://app.codecov.io/gh/apache/texera/commit/33cc5d16096aa05095f483ee8724e799c95d7898?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [frontend](https://app.codecov.io/gh/apache/texera/pull/7243/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.53% <ø> (ø)` | | Carriedforward from [33cc5d1](https://app.codecov.io/gh/apache/texera/commit/33cc5d16096aa05095f483ee8724e799c95d7898?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [notebook-migration-service](https://app.codecov.io/gh/apache/texera/pull/7243/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.89% <ø> (ø)` | | |
   | [pyamber](https://app.codecov.io/gh/apache/texera/pull/7243/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `97.36% <ø> (ø)` | | Carriedforward from [33cc5d1](https://app.codecov.io/gh/apache/texera/commit/33cc5d16096aa05095f483ee8724e799c95d7898?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/7243/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `26.31% <ø> (ø)` | | Carriedforward from [33cc5d1](https://app.codecov.io/gh/apache/texera/commit/33cc5d16096aa05095f483ee8724e799c95d7898?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   *This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/texera/pull/7243?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): validate base64 in HF binary-column fallback … [texera]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7243:
URL: https://github.com/apache/texera/pull/7243#issuecomment-5152643245

   <!-- backport-auto-label-report -->
   ### Backport auto-label report
   
   This `fix:` PR was checked against each actively-supported release branch. `release/*` labels drive the post-merge backport, so add or remove one to change where this fix lands.
   
   | Release branch | Analysis |
   | --- | --- |
   | ⚠️ `release/v1.2` | Not labeled automatically — none of the files this PR modifies exist on this branch (`common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/HuggingFaceCodegenBase.scala`, `common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala`). The fix may target code that isn't on this release, or the files were moved/renamed after the branch was cut. **Please check and add `release/v1.2` by hand if this fix should be backported here.** |
   
   _[Auto-label run](https://github.com/apache/texera/actions/runs/30711195442)._


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): validate base64 in HF binary-column fallback … [texera]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7243:
URL: https://github.com/apache/texera/pull/7243#issuecomment-5152659705

   <!-- texera-benchmarks-comment -->
   ## ⚠️ Benchmark changes need a look
   
   🟢 2 better · 🔴 6 worse · ⚪ 7 noise (<±5%) · 0 without baseline
   
   > Compared against main `33cc5d1` benchmarked on **this same runner**, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.
   
   [Dashboard](https://apache.github.io/texera/dev/bench/) · [Run](https://github.com/apache/texera/actions/runs/30711197153)
   
   | | config | throughput | MB/s | latency | max Δ latest / 7d |
   |---|---|---:|---:|---:|---:|
   | 🟢 | bs=10 sw=10 sl=64 | 648 | 0.396 | 15,487/18,990/18,990 us | 🟢 -15.8% / 🔴 +24.2% |
   | 🔴 | bs=100 sw=10 sl=64 | 1,192 | 0.728 | 82,819/107,204/107,204 us | 🔴 +21.6% / 🟢 +19.6% |
   | 🔴 | bs=1000 sw=10 sl=64 | 1,509 | 0.921 | 665,845/730,229/730,229 us | 🔴 +6.2% / 🟢 +47.4% |
   
   <details><summary>Baseline details</summary>
   
   Latest main `33cc5d1` from same runner
   
   | config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
   |---|---|---:|---:|---:|---:|---:|
   | bs=10 sw=10 sl=64 | throughput | 648 tuples/sec | 627 tuples/sec | 782.98 tuples/sec | +3.3% | -17.2% |
   | bs=10 sw=10 sl=64 | MB/s | 0.396 MB/s | 0.383 MB/s | 0.478 MB/s | +3.4% | -17.1% |
   | bs=10 sw=10 sl=64 | p50 | 15,487 us | 15,900 us | 12,467 us | -2.6% | +24.2% |
   | bs=10 sw=10 sl=64 | p95 | 18,990 us | 22,553 us | 15,695 us | -15.8% | +21.0% |
   | bs=10 sw=10 sl=64 | p99 | 18,990 us | 22,553 us | 18,950 us | -15.8% | +0.2% |
   | bs=100 sw=10 sl=64 | throughput | 1,192 tuples/sec | 1,384 tuples/sec | 996.92 tuples/sec | -13.9% | +19.6% |
   | bs=100 sw=10 sl=64 | MB/s | 0.728 MB/s | 0.844 MB/s | 0.608 MB/s | -13.7% | +19.6% |
   | bs=100 sw=10 sl=64 | p50 | 82,819 us | 71,384 us | 100,899 us | +16.0% | -17.9% |
   | bs=100 sw=10 sl=64 | p95 | 107,204 us | 88,139 us | 107,385 us | +21.6% | -0.2% |
   | bs=100 sw=10 sl=64 | p99 | 107,204 us | 88,139 us | 115,226 us | +21.6% | -7.0% |
   | bs=1000 sw=10 sl=64 | throughput | 1,509 tuples/sec | 1,588 tuples/sec | 1,024 tuples/sec | -5.0% | +47.4% |
   | bs=1000 sw=10 sl=64 | MB/s | 0.921 MB/s | 0.969 MB/s | 0.625 MB/s | -5.0% | +47.4% |
   | bs=1000 sw=10 sl=64 | p50 | 665,845 us | 627,160 us | 989,300 us | +6.2% | -32.7% |
   | bs=1000 sw=10 sl=64 | p95 | 730,229 us | 697,069 us | 1,031,689 us | +4.8% | -29.2% |
   | bs=1000 sw=10 sl=64 | p99 | 730,229 us | 697,069 us | 1,058,394 us | +4.8% | -31.0% |
   
   </details>
   
   <details><summary>Raw CSV</summary>
   
   ```csv
   config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
   0,10,10,64,20,308.57,200,128000,648,0.396,15487.04,18989.69,18989.69
   1,100,10,64,20,1677.48,2000,1280000,1192,0.728,82819.19,107204.45,107204.45
   2,1000,10,64,20,13250.38,20000,12800000,1509,0.921,665844.80,730229.45,730229.45
   ```
   
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): validate base64 in HF binary-column fallback … [texera]

Posted by "xuang7 (via GitHub)" <gi...@apache.org>.
xuang7 merged PR #7243:
URL: https://github.com/apache/texera/pull/7243


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(workflow-operator): validate base64 in HF binary-column fallback … [texera]

Posted by "PG1204 (via GitHub)" <gi...@apache.org>.
PG1204 commented on PR #7243:
URL: https://github.com/apache/texera/pull/7243#issuecomment-5172972833

   /request-review @xuang7 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org