You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "nsivabalan (via GitHub)" <gi...@apache.org> on 2026/04/02 06:23:41 UTC
[PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
nsivabalan opened a new pull request, #18447:
URL: https://github.com/apache/hudi/pull/18447
### Describe the issue this Pull Request addresses
<!-- Either describe the issue inline here with motivation behind the changes
(or) link to an issue by including `Closes #<issue-number>` for context.
If this PR includes changes to the storage format, public APIs,
or has breaking changes, use `!` (e.g., feat!: ...) -->
### Summary and Changelog
<!-- Short, plain-English summary of what users gain or what changed in behavior.
Followed by a detailed log of all the changes. Highlight if any code was copied. -->
### Impact
<!-- Describe any public API or user-facing feature change or any performance impact. -->
### Risk Level
<!-- Accepted values: none, low, medium or high. Other than `none`, explain the risk.
If medium or high, explain what verification was done to mitigate the risks. -->
### Documentation Update
<!-- Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".
- The config description must be updated if new configs are added or the default value of the configs are changed.
- Any new feature or user-facing change requires updating the Hudi website. Please follow the
[instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website. -->
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175241641
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.54%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 68.54% +2.44%
- Complexity 22535 28046 +5511
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 92397 +18929
- Misses 31449 35116 +3667
- Partials 6235 7290 +1055
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.70% <77.08%> (-0.07%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.64% <77.08%> (+0.34%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `74.50% <50.00%> (-0.20%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [828 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175147535
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.13%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 19 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|2|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
=============================================
- Coverage 66.09% 48.13% -17.96%
+ Complexity 22535 16414 -6121
=============================================
Files 1995 1999 +4
Lines 111152 111573 +421
Branches 14042 14127 +85
=============================================
- Hits 73468 53709 -19759
- Misses 31449 52232 +20783
+ Partials 6235 5632 -603
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.27% <50.00%> (-15.50%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.67% <77.08%> (-12.63%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `63.92% <50.00%> (-10.79%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [904 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175145647
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.10%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 20 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|1|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
=============================================
- Coverage 66.09% 48.10% -18.00%
+ Complexity 22535 16397 -6138
=============================================
Files 1995 1999 +4
Lines 111152 111573 +421
Branches 14042 14127 +85
=============================================
- Hits 73468 53668 -19800
- Misses 31449 52268 +20819
+ Partials 6235 5637 -598
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.68% <50.00%> (-15.09%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.67% <77.08%> (-12.63%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `63.92% <50.00%> (-10.79%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [905 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175185586
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.83%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
- Coverage 66.09% 65.83% -0.27%
- Complexity 22535 26820 +4285
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 88741 +15273
- Misses 31449 38678 +7229
- Partials 6235 7384 +1149
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.46% <66.66%> (-16.31%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.09% <77.08%> (-0.21%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `73.33% <50.00%> (-1.38%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [1107 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175028446
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 44.94%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 27 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-client-hadoop-common|1|0|
>|spark-scala-tests|10|0|
>|spark-java-tests|15|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
=============================================
- Coverage 66.09% 44.94% -21.16%
+ Complexity 22535 8476 -14059
=============================================
Files 1995 1195 -800
Lines 111152 61794 -49358
Branches 14042 6637 -7405
=============================================
- Hits 73468 27772 -45696
+ Misses 31449 30994 -455
+ Partials 6235 3028 -3207
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
[see 1556 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(spark): Use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #18447:
URL: https://github.com/apache/hudi/pull/18447#discussion_r3034570850
##########
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/hudi/analysis/Spark33HoodiePruneFileSourcePartitions.scala:
##########
@@ -0,0 +1,133 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hudi.analysis
+
+import org.apache.hudi.{HoodieBaseRelation, HoodieFileIndex}
+import org.apache.hudi.SparkAdapterSupport.sparkAdapter
+
+import org.apache.spark.sql.SparkSession
+import org.apache.spark.sql.catalyst.catalog.CatalogStatistics
+import org.apache.spark.sql.catalyst.expressions.{And, AttributeReference, AttributeSet, Expression, ExpressionSet, NamedExpression, PredicateHelper, SubqueryExpression}
+import org.apache.spark.sql.catalyst.planning.ScanOperation
+import org.apache.spark.sql.catalyst.plans.logical.{Filter, LeafNode, LogicalPlan, Project}
+import org.apache.spark.sql.catalyst.plans.logical.statsEstimation.FilterEstimation
+import org.apache.spark.sql.catalyst.rules.Rule
+import org.apache.spark.sql.execution.datasources.{HadoopFsRelation, LogicalRelation}
+import org.apache.spark.sql.hudi.analysis.Spark33HoodiePruneFileSourcePartitions.{exprUtils, getPartitionFiltersAndDataFilters, rebuildPhysicalOperation, HoodieRelationMatcher}
+import org.apache.spark.sql.sources.BaseRelation
+import org.apache.spark.sql.types.StructType
+
+/**
+ * Prune the partitions of Hudi table based relations by the means of pushing down the
+ * partition filters
+ *
+ * NOTE: In Spark 3.3 and earlier, ScanOperation and PhysicalOperation had different behaviors
+ * for filter pushdown and partition pruning. ScanOperation provides better compatibility
+ * with Spark's internal query optimization in 3.3.
+ *
+ * NOTE: SPARK-39764 unified PhysicalOperation and ScanOperation in Spark 3.4+, making this
+ * version-specific implementation necessary only for Spark 3.3.
+ *
+ * @see [[Spark3HoodiePruneFileSourcePartitions]] for Spark 3.4 and 3.5
+ * @see [[Spark4HoodiePruneFileSourcePartitions]] for Spark 4.0+
+ */
+case class Spark33HoodiePruneFileSourcePartitions(spark: SparkSession) extends Rule[LogicalPlan] {
+
+ override def apply(plan: LogicalPlan): LogicalPlan = plan transformDown {
+ case op @ ScanOperation(projects, filters, lr @ LogicalRelation(HoodieRelationMatcher(fileIndex), _, _, _))
+ if !fileIndex.hasPredicatesPushedDown =>
+
+ val deterministicFilters = filters.filter(f => f.deterministic && !SubqueryExpression.hasSubquery(f))
+ val normalizedFilters = exprUtils.normalizeExprs(deterministicFilters, lr.output)
+
+ val (partitionPruningFilters, dataFilters) =
+ getPartitionFiltersAndDataFilters(fileIndex.partitionSchema, normalizedFilters)
+
+ // [[HudiFileIndex]] is a caching one, therefore we don't need to reconstruct new relation,
+ // instead we simply just refresh the index and update the stats
+ fileIndex.filterFileSlices(dataFilters, partitionPruningFilters, isPartitionPruned = true)
+
+ if (partitionPruningFilters.nonEmpty) {
+ // Change table stats based on the sizeInBytes of pruned files
+ val filteredStats = FilterEstimation(Filter(partitionPruningFilters.reduce(And), lr)).estimate
+ val colStats = filteredStats.map {
Review Comment:
🤖 `fileIndex.filterFileSlices(...)` is called unconditionally on line 68 as a side effect (mutating the cached file index), but when `partitionPruningFilters` is empty, line 84 returns the original `op` without rebuilding the plan. This means data filters still get pushed down into the index even when no partition pruning occurs, and no stats update happens. This matches the existing Spark 3.4+ code so it may be intentional — but worth confirming: is it intended for `filterFileSlices` to mutate the index even when we return the original unmodified plan node?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4174985623
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "148f392f53ee312c0ebb9d52b29832566d7683d6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13023",
"triggerID" : "148f392f53ee312c0ebb9d52b29832566d7683d6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 148f392f53ee312c0ebb9d52b29832566d7683d6 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13023)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175110786
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 53.99%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 26 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-scala-tests|10|0|
>|spark-java-tests|15|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
=============================================
- Coverage 66.09% 53.99% -12.11%
+ Complexity 22535 12246 -10289
=============================================
Files 1995 1424 -571
Lines 111152 70704 -40448
Branches 14042 7951 -6091
=============================================
- Hits 73468 38177 -35291
+ Misses 31449 29113 -2336
+ Partials 6235 3414 -2821
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
[see 1289 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175201584
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.43%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 67.43% +1.34%
- Complexity 22535 27563 +5028
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 90910 +17442
- Misses 31449 36595 +5146
- Partials 6235 7298 +1063
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (-3.80%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.27% <77.08%> (-0.03%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `73.33% <50.00%> (-1.38%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [969 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4174973746
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "148f392f53ee312c0ebb9d52b29832566d7683d6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "148f392f53ee312c0ebb9d52b29832566d7683d6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 148f392f53ee312c0ebb9d52b29832566d7683d6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175154517
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.20%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 17 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|4|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
=============================================
- Coverage 66.09% 48.20% -17.90%
+ Complexity 22535 16435 -6100
=============================================
Files 1995 1999 +4
Lines 111152 111573 +421
Branches 14042 14127 +85
=============================================
- Hits 73468 53785 -19683
- Misses 31449 52170 +20721
+ Partials 6235 5618 -617
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.46% <66.66%> (-16.31%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.67% <77.08%> (-12.63%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `63.92% <50.00%> (-10.79%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [904 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(spark): Use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #18447:
URL: https://github.com/apache/hudi/pull/18447#discussion_r3034570850
##########
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/hudi/analysis/Spark33HoodiePruneFileSourcePartitions.scala:
##########
@@ -0,0 +1,133 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hudi.analysis
+
+import org.apache.hudi.{HoodieBaseRelation, HoodieFileIndex}
+import org.apache.hudi.SparkAdapterSupport.sparkAdapter
+
+import org.apache.spark.sql.SparkSession
+import org.apache.spark.sql.catalyst.catalog.CatalogStatistics
+import org.apache.spark.sql.catalyst.expressions.{And, AttributeReference, AttributeSet, Expression, ExpressionSet, NamedExpression, PredicateHelper, SubqueryExpression}
+import org.apache.spark.sql.catalyst.planning.ScanOperation
+import org.apache.spark.sql.catalyst.plans.logical.{Filter, LeafNode, LogicalPlan, Project}
+import org.apache.spark.sql.catalyst.plans.logical.statsEstimation.FilterEstimation
+import org.apache.spark.sql.catalyst.rules.Rule
+import org.apache.spark.sql.execution.datasources.{HadoopFsRelation, LogicalRelation}
+import org.apache.spark.sql.hudi.analysis.Spark33HoodiePruneFileSourcePartitions.{exprUtils, getPartitionFiltersAndDataFilters, rebuildPhysicalOperation, HoodieRelationMatcher}
+import org.apache.spark.sql.sources.BaseRelation
+import org.apache.spark.sql.types.StructType
+
+/**
+ * Prune the partitions of Hudi table based relations by the means of pushing down the
+ * partition filters
+ *
+ * NOTE: In Spark 3.3 and earlier, ScanOperation and PhysicalOperation had different behaviors
+ * for filter pushdown and partition pruning. ScanOperation provides better compatibility
+ * with Spark's internal query optimization in 3.3.
+ *
+ * NOTE: SPARK-39764 unified PhysicalOperation and ScanOperation in Spark 3.4+, making this
+ * version-specific implementation necessary only for Spark 3.3.
+ *
+ * @see [[Spark3HoodiePruneFileSourcePartitions]] for Spark 3.4 and 3.5
+ * @see [[Spark4HoodiePruneFileSourcePartitions]] for Spark 4.0+
+ */
+case class Spark33HoodiePruneFileSourcePartitions(spark: SparkSession) extends Rule[LogicalPlan] {
+
+ override def apply(plan: LogicalPlan): LogicalPlan = plan transformDown {
+ case op @ ScanOperation(projects, filters, lr @ LogicalRelation(HoodieRelationMatcher(fileIndex), _, _, _))
+ if !fileIndex.hasPredicatesPushedDown =>
+
+ val deterministicFilters = filters.filter(f => f.deterministic && !SubqueryExpression.hasSubquery(f))
+ val normalizedFilters = exprUtils.normalizeExprs(deterministicFilters, lr.output)
+
+ val (partitionPruningFilters, dataFilters) =
+ getPartitionFiltersAndDataFilters(fileIndex.partitionSchema, normalizedFilters)
+
+ // [[HudiFileIndex]] is a caching one, therefore we don't need to reconstruct new relation,
+ // instead we simply just refresh the index and update the stats
+ fileIndex.filterFileSlices(dataFilters, partitionPruningFilters, isPartitionPruned = true)
+
+ if (partitionPruningFilters.nonEmpty) {
+ // Change table stats based on the sizeInBytes of pruned files
+ val filteredStats = FilterEstimation(Filter(partitionPruningFilters.reduce(And), lr)).estimate
+ val colStats = filteredStats.map {
Review Comment:
🤖 `fileIndex.filterFileSlices(...)` is called unconditionally on line 68 as a side effect (mutating the cached file index), but when `partitionPruningFilters` is empty, line 84 returns the original `op` without rebuilding the plan. This means data filters still get pushed down into the index even when no partition pruning occurs, and no stats update happens. This matches the existing Spark 3.4+ code so it may be intentional — but worth confirming: is it intended for `filterFileSlices` to mutate the index even when we return the original unmodified plan node?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175169478
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 61.25%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
- Coverage 66.09% 61.25% -4.85%
- Complexity 22535 24960 +2425
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 82574 +9106
- Misses 31449 45299 +13850
- Partials 6235 6930 +695
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.46% <66.66%> (-16.31%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.67% <77.08%> (-12.63%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `63.92% <50.00%> (-10.79%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [1263 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175230564
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.70%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 67.70% +1.61%
- Complexity 22535 27704 +5169
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 91271 +17803
- Misses 31449 36173 +4724
- Partials 6235 7359 +1124
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.06% <66.66%> (-3.71%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.64% <77.08%> (+0.34%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `74.50% <50.00%> (-0.20%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [935 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175237967
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.50%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 68.50% +2.40%
- Complexity 22535 28017 +5482
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 92349 +18881
- Misses 31449 35156 +3707
- Partials 6235 7298 +1063
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.61% <77.08%> (-0.16%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.64% <77.08%> (+0.34%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `74.50% <50.00%> (-0.20%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [837 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175235622
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.48%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 68.48% +2.39%
- Complexity 22535 28011 +5476
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 92324 +18856
- Misses 31449 35180 +3731
- Partials 6235 7299 +1064
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.58% <66.66%> (-0.19%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.64% <77.08%> (+0.34%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `74.50% <50.00%> (-0.20%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [841 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175205889
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.58%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 67.58% +1.48%
- Complexity 22535 27635 +5100
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 91102 +17634
- Misses 31449 36380 +4931
- Partials 6235 7321 +1086
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (-3.80%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.48% <77.08%> (+0.18%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `74.50% <50.00%> (-0.20%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [947 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175283425
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.54%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 68.54% +2.45%
- Complexity 22535 28046 +5511
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 92406 +18938
- Misses 31449 35107 +3658
- Partials 6235 7290 +1055
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.78% <77.08%> (+<0.01%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.64% <77.08%> (+0.34%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `74.50% <50.00%> (-0.20%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [827 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175298361
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.55%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 68.55% +2.45%
- Complexity 22535 28048 +5513
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 92408 +18940
- Misses 31449 35106 +3657
- Partials 6235 7289 +1054
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.78% <77.08%> (+0.01%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.64% <77.08%> (+0.34%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `74.50% <50.00%> (-0.20%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [826 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175131340
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.77%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 23 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-scala-tests|10|3|
>|spark-java-tests|15|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
=============================================
- Coverage 66.09% 45.77% -20.33%
+ Complexity 22535 15281 -7254
=============================================
Files 1995 1970 -25
Lines 111152 108960 -2192
Branches 14042 13565 -477
=============================================
- Hits 73468 49875 -23593
- Misses 31449 53824 +22375
+ Partials 6235 5261 -974
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.80% <77.08%> (-12.50%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `59.60% <50.00%> (-15.10%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [983 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175131909
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.77%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 23 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-scala-tests|10|3|
>|spark-java-tests|15|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
=============================================
- Coverage 66.09% 45.77% -20.33%
+ Complexity 22535 15281 -7254
=============================================
Files 1995 1970 -25
Lines 111152 108960 -2192
Branches 14042 13565 -477
=============================================
- Hits 73468 49875 -23593
- Misses 31449 53824 +22375
+ Partials 6235 5261 -974
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.80% <77.08%> (-12.50%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `59.60% <50.00%> (-15.10%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [983 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175130798
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.77%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 23 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-scala-tests|10|3|
>|spark-java-tests|15|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
=============================================
- Coverage 66.09% 45.77% -20.33%
+ Complexity 22535 15281 -7254
=============================================
Files 1995 1970 -25
Lines 111152 108960 -2192
Branches 14042 13565 -477
=============================================
- Hits 73468 49875 -23593
- Misses 31449 53824 +22375
+ Partials 6235 5261 -974
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.80% <77.08%> (-12.50%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `59.60% <50.00%> (-15.10%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [983 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175219893
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.58%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 67.58% +1.48%
- Complexity 22535 27635 +5100
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 91104 +17636
- Misses 31449 36380 +4931
- Partials 6235 7319 +1084
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.97% <66.66%> (-3.80%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.48% <77.08%> (+0.19%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `74.50% <50.00%> (-0.20%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [947 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175225000
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.70%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 67.70% +1.60%
- Complexity 22535 27704 +5169
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 91270 +17802
- Misses 31449 36174 +4725
- Partials 6235 7359 +1124
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.06% <66.66%> (-3.71%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.64% <77.08%> (+0.34%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `74.50% <50.00%> (-0.20%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [936 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175149373
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.17%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 18 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|3|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
=============================================
- Coverage 66.09% 48.17% -17.93%
+ Complexity 22535 16425 -6110
=============================================
Files 1995 1999 +4
Lines 111152 111573 +421
Branches 14042 14127 +85
=============================================
- Hits 73468 53752 -19716
- Misses 31449 52199 +20750
+ Partials 6235 5622 -613
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.89% <66.66%> (-15.88%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.67% <77.08%> (-12.63%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `63.92% <50.00%> (-10.79%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [904 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175200094
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.94%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
- Coverage 66.09% 65.94% -0.15%
- Complexity 22535 26887 +4352
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 88901 +15433
- Misses 31449 38471 +7022
- Partials 6235 7431 +1196
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.46% <66.66%> (-16.31%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.27% <77.08%> (-0.03%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `73.33% <50.00%> (-1.38%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [1098 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175316578
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "148f392f53ee312c0ebb9d52b29832566d7683d6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13023",
"triggerID" : "148f392f53ee312c0ebb9d52b29832566d7683d6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 148f392f53ee312c0ebb9d52b29832566d7683d6 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=13023)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175303300
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.55%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 68.55% +2.45%
- Complexity 22535 28048 +5513
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 92408 +18940
- Misses 31449 35106 +3657
- Partials 6235 7289 +1054
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.81% <77.08%> (+0.04%)` | :arrow_up: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.64% <77.08%> (+0.34%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `74.50% <50.00%> (-0.20%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [826 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175143362
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.64%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 21 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-scala-tests|10|4|
>|spark-java-tests|15|1|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
=============================================
- Coverage 66.09% 48.64% -17.45%
+ Complexity 22535 16187 -6348
=============================================
Files 1995 1970 -25
Lines 111152 108965 -2187
Branches 14042 13571 -471
=============================================
- Hits 73468 53009 -20459
- Misses 31449 50482 +19033
+ Partials 6235 5474 -761
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.68% <50.00%> (-15.09%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.80% <77.08%> (-12.50%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `61.56% <50.00%> (-13.14%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [923 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(spark): Use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed pull request #18447: feat(spark): Use ScanOperation for Spark 3.3 Partition Pruning
URL: https://github.com/apache/hudi/pull/18447
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175270929
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 68.54%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
+ Coverage 66.09% 68.54% +2.44%
- Complexity 22535 28046 +5511
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 92397 +18929
- Misses 31449 35116 +3667
- Partials 6235 7290 +1055
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.73% <77.08%> (-0.04%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.64% <77.08%> (+0.34%)` | :arrow_up: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `74.50% <50.00%> (-0.20%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [828 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175141207
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.77%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 22 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-scala-tests|10|4|
>|spark-java-tests|15|0|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
=============================================
- Coverage 66.09% 45.77% -20.32%
+ Complexity 22535 15282 -7253
=============================================
Files 1995 1970 -25
Lines 111152 108965 -2187
Branches 14042 13571 -471
=============================================
- Hits 73468 49881 -23587
- Misses 31449 53825 +22376
+ Partials 6235 5259 -976
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.80% <77.08%> (-12.50%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `60.00% <50.00%> (-14.71%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [981 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175168369
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 61.25%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
- Coverage 66.09% 61.25% -4.85%
- Complexity 22535 24960 +2425
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 82574 +9106
- Misses 31449 45299 +13850
- Partials 6235 6930 +695
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.46% <66.66%> (-16.31%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.67% <77.08%> (-12.63%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.36% <50.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `63.92% <50.00%> (-10.79%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [1263 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: use ScanOperation for Spark 3.3 Partition Pruning [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18447:
URL: https://github.com/apache/hudi/pull/18447#issuecomment-4175163450
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `77.08333%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 58.74%. Comparing base ([`e270e25`](https://app.codecov.io/gh/apache/hudi/commit/e270e256bac65961e4be9ab24061925337889514?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`148f392`](https://app.codecov.io/gh/apache/hudi/commit/148f392f53ee312c0ebb9d52b29832566d7683d6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | 78.26% | [3 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | 50.00% | [0 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (e270e25) and HEAD (148f392). Click for more details.
>
> <details><summary>HEAD has 17 uploads less than BASE</summary>
>
>| Flag | BASE (e270e25) | HEAD (148f392) |
>|------|------|------|
>|spark-scala-tests|10|5|
>|spark-java-tests|15|4|
>|utilities|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18447 +/- ##
============================================
- Coverage 66.09% 58.74% -7.36%
- Complexity 22535 23993 +1458
============================================
Files 1995 2445 +450
Lines 111152 134803 +23651
Branches 14042 16302 +2260
============================================
+ Hits 73468 79184 +5716
- Misses 31449 48841 +17392
- Partials 6235 6778 +543
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [common-and-other-modules](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.36% <50.00%> (?)` | |
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.94% <ø> (-0.04%)` | :arrow_down: |
| [spark-client-hadoop-common](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.45% <ø> (+0.05%)` | :arrow_up: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.46% <66.66%> (-16.31%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.67% <77.08%> (-12.63%)` | :arrow_down: |
| [utilities](https://app.codecov.io/gh/apache/hudi/pull/18447/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18447?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/spark/sql/hudi/analysis/HoodieAnalysis.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FHoodieAnalysis.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9zcGFyay9zcWwvaHVkaS9hbmFseXNpcy9Ib29kaWVBbmFseXNpcy5zY2FsYQ==) | `63.92% <50.00%> (-10.79%)` | :arrow_down: |
| [...lysis/Spark33HoodiePruneFileSourcePartitions.scala](https://app.codecov.io/gh/apache/hudi/pull/18447?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.3.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fhudi%2Fanalysis%2FSpark33HoodiePruneFileSourcePartitions.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjMueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9odWRpL2FuYWx5c2lzL1NwYXJrMzNIb29kaWVQcnVuZUZpbGVTb3VyY2VQYXJ0aXRpb25zLnNjYWxh) | `78.26% <78.26%> (ø)` | |
... and [1356 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18447/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org