You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2026/09/27 11:26:58 UTC
[PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
yihua opened a new pull request, #20102:
URL: https://github.com/apache/hudi/pull/20102
### Describe the issue this Pull Request addresses
closes #20100
part of #20064
For every parquet base file, the Spark readers copy the Hadoop conf three times and render the requested schema to JSON three times, although those keys depend only on the requested schema. Every task also converts the table schema for timestamp repair even when repair is off, the Spark 3.x readers walk every footer for shredded variants, and the vectorized reader allocates a full batch even for small files.
### Summary and Changelog
`SparkParquetReaderBase` prepares a `ParquetReadConf` (a `JobConf` with the requested schema keys) once per scan and requested schema when the caller passes a `SharedScanStorageConfiguration`, as the file group reader format now does for base files; other callers get one copy per file instead of three. `ParquetSchemaEvolutionUtils.getHadoopAttemptConf` (was `getHadoopConfClone`) copies the conf only when the file needs its own requested schema or a pushed filter, so the shared conf is never modified. The Spark 3.x readers skip the variant walk when no requested struct can be an unshredded variant, vectorized batches on Spark 3.5+ are sized to the split's rows, and the timestamp repair schema is converted only for tables with a timestamp-millis field. New tests check that a COW scan reads all base files with one conf and that shared and per-file confs return the same rows, including with pushed filters, type changes and 8 concurrent threads.
### Impact
Lower per-file CPU on Spark parquet base file reads. No API, config or output change.
### Risk Level
low. The files of a scan share a read-only `JobConf`, as vanilla Spark already does for footer reads; per-file keys go on a copy, and a concurrent test checks the shared conf stays unchanged.
Merging with #20079 (conflicts in `ParquetSchemaEvolutionUtils`, `SparkParquetReaderBase` and the six readers): take this PR's `getHadoopAttemptConf` and drop #20079's in-place `val hadoopAttemptConf = readConf` (`ParquetSchemaEvolutionUtils.scala:89`). With a shared conf, that write leaks one file's pushed filter to other files and silently drops their rows. Keep `TestSparkParquetReaderBase#testAttemptConfIsTheSharedConfUnlessTheFileNeedsItsOwnKeys` and `TestSharedScanParquetReadConf` through the merge; they fail on the wrong resolution.
This PR also conflicts with #20090 (`ParquetSchemaEvolutionUtils`, the readers, `TestBasicSchemaEvolution`) and #20077 (`HoodieFileGroupReaderBasedFileFormat`, whose two edits move to `HoodieFileGroupReaderFunction`); whichever merges second rebases.
### Documentation Update
none
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #20102:
URL: https://github.com/apache/hudi/pull/20102#issuecomment-5855524071
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/20102?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 80.40%. Comparing base ([`ead194c`](https://app.codecov.io/gh/apache/hudi/commit/ead194cce54829fae9113689a2f28772d0d76045?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`40886e1`](https://app.codecov.io/gh/apache/hudi/commit/40886e19699a6768d97cca6f15d96ca2504d65e5?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #20102 +/- ##
============================================
- Coverage 80.40% 80.40% -0.01%
+ Complexity 34874 34867 -7
============================================
Files 2546 2546
Lines 142889 142863 -26
Branches 17374 17372 -2
============================================
- Hits 114895 114871 -24
Misses 20082 20082
+ Partials 7912 7910 -2
```
| [Components](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hudi-common](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.97% <ø> (ø)` | |
| [hudi-client](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.45% <ø> (ø)` | |
| [hudi-flink](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.74% <ø> (ø)` | |
| [hudi-spark-datasource](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `73.83% <ø> (-0.03%)` | :arrow_down: |
| [hudi-utilities](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.17% <ø> (ø)` | |
| [hudi-cli](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.05% <ø> (ø)` | |
| [hudi-hadoop](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.99% <ø> (ø)` | |
| [hudi-sync](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `76.02% <ø> (ø)` | |
| [hudi-io](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.50% <ø> (ø)` | |
| [hudi-timeline-service](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.06% <ø> (ø)` | |
| [hudi-cloud](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.00% <ø> (ø)` | |
| [hudi-kafka-connect](https://app.codecov.io/gh/apache/hudi/pull/20102/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.20% <ø> (ø)` | |
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/20102/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/20102/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.98% <ø> (ø)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/20102?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...parquet/HoodieFileGroupReaderBasedFileFormat.scala](https://app.codecov.io/gh/apache/hudi/pull/20102?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark-common%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fexecution%2Fdatasources%2Fparquet%2FHoodieFileGroupReaderBasedFileFormat.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmstY29tbW9uL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsvc3FsL2V4ZWN1dGlvbi9kYXRhc291cmNlcy9wYXJxdWV0L0hvb2RpZUZpbGVHcm91cFJlYWRlckJhc2VkRmlsZUZvcm1hdC5zY2FsYQ==) | `84.08% <ø> (-0.28%)` | :arrow_down: |
| [...asources/parquet/ParquetSchemaEvolutionUtils.scala](https://app.codecov.io/gh/apache/hudi/pull/20102?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark-common%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fexecution%2Fdatasources%2Fparquet%2FParquetSchemaEvolutionUtils.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmstY29tbW9uL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsvc3FsL2V4ZWN1dGlvbi9kYXRhc291cmNlcy9wYXJxdWV0L1BhcnF1ZXRTY2hlbWFFdm9sdXRpb25VdGlscy5zY2FsYQ==) | `87.43% <ø> (+0.33%)` | :arrow_up: |
| [...n/datasources/parquet/SparkParquetReaderBase.scala](https://app.codecov.io/gh/apache/hudi/pull/20102?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark-common%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fexecution%2Fdatasources%2Fparquet%2FSparkParquetReaderBase.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmstY29tbW9uL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvc3Bhcmsvc3FsL2V4ZWN1dGlvbi9kYXRhc291cmNlcy9wYXJxdWV0L1NwYXJrUGFycXVldFJlYWRlckJhc2Uuc2NhbGE=) | `100.00% <ø> (+6.66%)` | :arrow_up: |
| [.../parquet/Spark3LegacyHoodieParquetFileFormat.scala](https://app.codecov.io/gh/apache/hudi/pull/20102?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3-common%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fexecution%2Fdatasources%2Fparquet%2FSpark3LegacyHoodieParquetFileFormat.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLWNvbW1vbi9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9leGVjdXRpb24vZGF0YXNvdXJjZXMvcGFycXVldC9TcGFyazNMZWdhY3lIb29kaWVQYXJxdWV0RmlsZUZvcm1hdC5zY2FsYQ==) | `0.00% <ø> (ø)` | |
| [...ion/datasources/parquet/Spark35ParquetReader.scala](https://app.codecov.io/gh/apache/hudi/pull/20102?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark3.5.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fexecution%2Fdatasources%2Fparquet%2FSpark35ParquetReader.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmszLjUueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9leGVjdXRpb24vZGF0YXNvdXJjZXMvcGFycXVldC9TcGFyazM1UGFycXVldFJlYWRlci5zY2FsYQ==) | `94.30% <ø> (-0.11%)` | :arrow_down: |
| [...parquet/Spark42LegacyHoodieParquetFileFormat.scala](https://app.codecov.io/gh/apache/hudi/pull/20102?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark4.2.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fexecution%2Fdatasources%2Fparquet%2FSpark42LegacyHoodieParquetFileFormat.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcms0LjIueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9leGVjdXRpb24vZGF0YXNvdXJjZXMvcGFycXVldC9TcGFyazQyTGVnYWN5SG9vZGllUGFycXVldEZpbGVGb3JtYXQuc2NhbGE=) | `87.62% <ø> (ø)` | |
| [...ion/datasources/parquet/Spark42ParquetReader.scala](https://app.codecov.io/gh/apache/hudi/pull/20102?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark4.2.x%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fsql%2Fexecution%2Fdatasources%2Fparquet%2FSpark42ParquetReader.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcms0LjIueC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL3NwYXJrL3NxbC9leGVjdXRpb24vZGF0YXNvdXJjZXMvcGFycXVldC9TcGFyazQyUGFycXVldFJlYWRlci5zY2FsYQ==) | `94.79% <ø> (-0.06%)` | :arrow_down: |
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous commented on PR #20102:
URL: https://github.com/apache/hudi/pull/20102#issuecomment-6701548614
Temporarily closing this PR while we debug a GitHub settings error on master (#20471); it will be reopened shortly, no action needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #20102:
URL: https://github.com/apache/hudi/pull/20102#issuecomment-5857099940
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17313",
"triggerID" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"triggerType" : "PUSH"
}, {
"hash" : "22367fde9645c4df16212c729f529f9e53a08a26",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17315",
"triggerID" : "22367fde9645c4df16212c729f529f9e53a08a26",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 22367fde9645c4df16212c729f529f9e53a08a26 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17315)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #20102:
URL: https://github.com/apache/hudi/pull/20102#issuecomment-5964956717
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17313",
"triggerID" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"triggerType" : "PUSH"
}, {
"hash" : "22367fde9645c4df16212c729f529f9e53a08a26",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17315",
"triggerID" : "22367fde9645c4df16212c729f529f9e53a08a26",
"triggerType" : "PUSH"
}, {
"hash" : "fadb152e4b552aee1cef1f57f56ecca26b7c299b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17436",
"triggerID" : "fadb152e4b552aee1cef1f57f56ecca26b7c299b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fadb152e4b552aee1cef1f57f56ecca26b7c299b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17436)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #20102:
URL: https://github.com/apache/hudi/pull/20102#issuecomment-5855512702
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17313",
"triggerID" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 40886e19699a6768d97cca6f15d96ca2504d65e5 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17313)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous closed pull request #20102: perf(spark): set scan-constant parquet read keys once per scan
URL: https://github.com/apache/hudi/pull/20102
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #20102:
URL: https://github.com/apache/hudi/pull/20102#issuecomment-5856271441
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17313",
"triggerID" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"triggerType" : "PUSH"
}, {
"hash" : "22367fde9645c4df16212c729f529f9e53a08a26",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17315",
"triggerID" : "22367fde9645c4df16212c729f529f9e53a08a26",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 40886e19699a6768d97cca6f15d96ca2504d65e5 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17313)
* 22367fde9645c4df16212c729f529f9e53a08a26 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17315)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #20102:
URL: https://github.com/apache/hudi/pull/20102#issuecomment-5963388279
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17313",
"triggerID" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"triggerType" : "PUSH"
}, {
"hash" : "22367fde9645c4df16212c729f529f9e53a08a26",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17315",
"triggerID" : "22367fde9645c4df16212c729f529f9e53a08a26",
"triggerType" : "PUSH"
}, {
"hash" : "fadb152e4b552aee1cef1f57f56ecca26b7c299b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fadb152e4b552aee1cef1f57f56ecca26b7c299b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 22367fde9645c4df16212c729f529f9e53a08a26 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17315)
* fadb152e4b552aee1cef1f57f56ecca26b7c299b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #20102:
URL: https://github.com/apache/hudi/pull/20102#issuecomment-5855462957
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 40886e19699a6768d97cca6f15d96ca2504d65e5 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #20102:
URL: https://github.com/apache/hudi/pull/20102#issuecomment-5856225178
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17313",
"triggerID" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"triggerType" : "PUSH"
}, {
"hash" : "22367fde9645c4df16212c729f529f9e53a08a26",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "22367fde9645c4df16212c729f529f9e53a08a26",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 40886e19699a6768d97cca6f15d96ca2504d65e5 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17313)
* 22367fde9645c4df16212c729f529f9e53a08a26 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(spark): set scan-constant parquet read keys once per scan [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #20102:
URL: https://github.com/apache/hudi/pull/20102#issuecomment-5963449772
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17313",
"triggerID" : "40886e19699a6768d97cca6f15d96ca2504d65e5",
"triggerType" : "PUSH"
}, {
"hash" : "22367fde9645c4df16212c729f529f9e53a08a26",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17315",
"triggerID" : "22367fde9645c4df16212c729f529f9e53a08a26",
"triggerType" : "PUSH"
}, {
"hash" : "fadb152e4b552aee1cef1f57f56ecca26b7c299b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17436",
"triggerID" : "fadb152e4b552aee1cef1f57f56ecca26b7c299b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 22367fde9645c4df16212c729f529f9e53a08a26 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17315)
* fadb152e4b552aee1cef1f57f56ecca26b7c299b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17436)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org