You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "wombatu-kun (via GitHub)" <gi...@apache.org> on 2026/02/20 07:33:59 UTC
[PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
wombatu-kun opened a new pull request, #18229:
URL: https://github.com/apache/hudi/pull/18229
### Describe the issue this Pull Request addresses
When using Hudi as a lookup join dimension table in Flink, the entire table is loaded into the cache on each reload. For large dimension tables that are partitioned by a slowly-changing attribute (e.g., date, region), this means reading and caching data from all partitions — including historical ones that are irrelevant to the running job. This can cause excessive memory pressure and slow cache warm-up times.
This PR introduces a new option lookup.partitions that allows users to declare which partitions should be loaded into the lookup join cache, so that only the relevant slice of the dimension table is materialized in memory.
### Summary and Changelog
Users can now limit the lookup join cache to a subset of partitions by specifying the `lookup.partitions` option via a query hint. This is analogous to the existing lookup.async option that controls the lookup execution mode.
Partitions are expressed as comma-separated `key=value pairs` (one partition per entry), with multiple partitions separated by ;:
hudi-flink-datasource/hudi-flink — FlinkOptions.java
Added LOOKUP_PARTITIONS (lookup.partitions) config option of type String with no default value.
hudi-flink-datasource/hudi-flink — HoodieTableSource.java
Modified getLookupRuntimeProvider() to build an optional partition pruner from lookup.partitions and pass it to the HoodieLookupTableReader supplier when the option is set.
Added buildLookupPartitionPruner(): reads lookup.partitions, delegates path parsing to PartitionPathParser, and wraps the result in a PartitionPruners.StaticPartitionPruner via the existing PartitionPruners.Builder API.
Added getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner): temporarily substitutes this.partitionPruner with the lookup-specific pruner before calling getBatchInputFormat(), then restores the original value. This allows the lookup path to reuse all existing batch input format logic (COW, MOR, read-optimized) without duplication.
hudi-common — PartitionPathParser.java
Added parseLookupPartitionPaths(String spec, List<String> partitionKeys, boolean hiveStyle): parses the lookup.partitions spec into a list of Hudi partition paths.
Validates that every key in the spec belongs to the table's declared partition key set; throws IllegalArgumentException with an informative message (including the list of valid keys) on unknown keys.
Validates that every token follows key=value format; throws IllegalArgumentException on bare values without a key.
Constructs paths in partitionKeys order regardless of the order keys appear in the spec, ensuring correct path generation for both Hive-style (key=value/key=value) and plain-value (value/value) layouts.
### Impact
When lookup.partitions is not configured — the feature is fully opt-in and backward-compatible. When configured, only the specified partitions are read during cache population; rows whose lookup keys resolve to an excluded partition will not match any cache entry and will produce null values on the dimension side of a LEFT lookup join.
Cache warm-up time and memory footprint are reduced proportionally to the fraction of partitions excluded.
### Risk Level
none
### Documentation Update
Documentation update is needed: to add new FlinkOption `lookup.partitions`.
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945591267
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949164439
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 51.81%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
>
> <details><summary>HEAD has 7 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
>|------|------|------|
>|spark-scala-tests|6|5|
>|spark-java-tests|9|3|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 51.81% -5.47%
+ Complexity 18532 16563 -1969
============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
============================================
- Hits 60797 55013 -5784
- Misses 39612 45248 +5636
- Partials 5723 5903 +180
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.22% <0.00%> (-14.20%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.42% <0.00%> (-0.09%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
... and [508 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944959660
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 39.70%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
>
> <details><summary>HEAD has 12 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (5b7100c) |
>|------|------|------|
>|spark-scala-tests|6|3|
>|spark-java-tests|9|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=============================================
- Coverage 57.28% 39.70% -17.58%
+ Complexity 18532 12574 -5958
=============================================
Files 1944 1944
Lines 106132 106157 +25
Branches 13118 13124 +6
=============================================
- Hits 60797 42153 -18644
- Misses 39612 58945 +19333
+ Partials 5723 5059 -664
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `17.20% <0.00%> (-9.27%)` | :arrow_down: |
... and [856 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949099690
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 39.70%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
>
> <details><summary>HEAD has 12 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
>|------|------|------|
>|spark-scala-tests|6|3|
>|spark-java-tests|9|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=============================================
- Coverage 57.28% 39.70% -17.58%
+ Complexity 18532 12575 -5957
=============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
=============================================
- Hits 60797 42155 -18642
- Misses 39612 58949 +19337
+ Partials 5723 5060 -663
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
... and [858 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949114686
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 43.42%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
>
> <details><summary>HEAD has 11 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
>|------|------|------|
>|spark-scala-tests|6|3|
>|spark-java-tests|9|1|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=============================================
- Coverage 57.28% 43.42% -13.86%
+ Complexity 18532 13807 -4725
=============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
=============================================
- Hits 60797 46104 -14693
- Misses 39612 54749 +15137
+ Partials 5723 5311 -412
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.08% <0.00%> (-13.35%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
... and [787 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949177596
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 55.53%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 55.53% -1.75%
+ Complexity 18532 17918 -614
============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
============================================
- Hits 60797 58960 -1837
- Misses 39612 41411 +1799
- Partials 5723 5793 +70
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.53% <0.00%> (-3.90%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
... and [235 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932188080
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8d6d98805ddabfa1da4ff4df6304984b37ec6982 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932194714
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8d6d98805ddabfa1da4ff4df6304984b37ec6982 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945199225
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1f1bb42055b7f176cb4d4d2bd614de433d99740a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215)
* 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932438384
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8d6d98805ddabfa1da4ff4df6304984b37ec6982 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147)
* 02979148bf26522bbb63b4128e06a516684e37cf Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932789247
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 02979148bf26522bbb63b4128e06a516684e37cf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945129845
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 57.27%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 57.27% -0.02%
- Complexity 18532 18536 +4
============================================
Files 1944 1944
Lines 106132 106157 +25
Branches 13118 13124 +6
============================================
+ Hits 60797 60798 +1
- Misses 39612 39639 +27
+ Partials 5723 5720 -3
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.42% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
... and [10 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "cshuo (via GitHub)" <gi...@apache.org>.
cshuo commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2844215651
##########
hudi-common/src/main/java/org/apache/hudi/common/table/PartitionPathParser.java:
##########
@@ -164,4 +171,53 @@ private static Object inferDateValue(
private static boolean isTimeBasedType(HoodieSchemaType type) {
return type == HoodieSchemaType.DATE || type == HoodieSchemaType.TIMESTAMP || type == HoodieSchemaType.TIME;
}
+
+ /**
+ * Parses the {@code lookup.partitions} option value into a list of Hudi partition paths.
+ *
+ * <p>The spec format is {@code key1=val1,key2=val2} per partition, with multiple partitions
+ * separated by {@code ;}. Example: {@code "dt=2024-01-01,region=us;dt=2024-01-02,region=eu"}.
+ *
+ * @param spec the raw option value
+ * @param partitionKeys ordered list of partition key names as defined in the table schema
+ * @param hiveStyle whether the table uses Hive-style partitioning ({@code key=value} directories)
+ * @return list of partition paths in the format used by Hudi's file index
+ * @throws IllegalArgumentException if any key in the spec is not a valid partition key,
+ * or if a key-value pair does not follow {@code key=value} format
+ */
+ public static List<String> parseLookupPartitionPaths(String spec, List<String> partitionKeys, boolean hiveStyle) {
Review Comment:
lookup.partitions currently allows partial key sets for multi-key partitioned tables (e.g. only year=2024 when partition keys are year,month).
For lookup pruning, this is risky because StaticPartitionPruner does exact partition-path matching. A partial path can silently fail to match real partition directories (e.g. year=2024/month=01), resulting in empty lookup cache for expected rows.
Could we enforce that each partition spec includes all declared partition keys (fail fast with IllegalArgumentException if any key is missing)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949242641
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 57.26%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 57.26% -0.03%
Complexity 18532 18532
============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
============================================
- Hits 60797 60791 -6
- Misses 39612 39646 +34
- Partials 5723 5727 +4
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.34% <0.00%> (-0.08%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
... and [12 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3958948075
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
}, {
"hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
"triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"triggerType" : "PUSH"
}, {
"hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
"triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"triggerType" : "PUSH"
}, {
"hash" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12293",
"triggerID" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f1ea3e2b59bfa373ccca2abf12004acb221e8639 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258)
* cbe77a8b82bd7b752a9364f75fa32e409b7539eb Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12293)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3958933488
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
}, {
"hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
"triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"triggerType" : "PUSH"
}, {
"hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
"triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"triggerType" : "PUSH"
}, {
"hash" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f1ea3e2b59bfa373ccca2abf12004acb221e8639 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258)
* cbe77a8b82bd7b752a9364f75fa32e409b7539eb UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945100796
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 57.26%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 57.26% -0.02%
- Complexity 18532 18534 +2
============================================
Files 1944 1944
Lines 106132 106157 +25
Branches 13118 13124 +6
============================================
- Hits 60797 60792 -5
- Misses 39612 39643 +31
+ Partials 5723 5722 -1
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.35% <0.00%> (-0.08%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
... and [10 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3948928754
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
}, {
"hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
"triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"triggerType" : "PUSH"
}, {
"hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219)
* 0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251)
* f1ea3e2b59bfa373ccca2abf12004acb221e8639 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3948919348
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
}, {
"hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
"triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219)
* 0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949194383
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
}, {
"hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
"triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"triggerType" : "PUSH"
}, {
"hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
"triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219)
* 0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251)
* f1ea3e2b59bfa373ccca2abf12004acb221e8639 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949247478
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 57.26%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 57.26% -0.03%
- Complexity 18532 18534 +2
============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
============================================
- Hits 60797 60793 -4
- Misses 39612 39645 +33
- Partials 5723 5726 +3
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.38% <0.00%> (-0.05%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
... and [13 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944941505
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 40.51%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
>
> <details><summary>HEAD has 14 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (5b7100c) |
>|------|------|------|
>|spark-scala-tests|6|1|
>|spark-java-tests|9|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=============================================
- Coverage 57.28% 40.51% -16.77%
+ Complexity 18532 12237 -6295
=============================================
Files 1944 1910 -34
Lines 106132 101406 -4726
Branches 13118 12026 -1092
=============================================
- Hits 60797 41087 -19710
- Misses 39612 55498 +15886
+ Partials 5723 4821 -902
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.64% <0.00%> (-11.87%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `17.20% <0.00%> (-9.27%)` | :arrow_down: |
... and [884 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945022855
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 51.69%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
>
> <details><summary>HEAD has 8 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (5b7100c) |
>|------|------|------|
>|spark-scala-tests|6|4|
>|spark-java-tests|9|3|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 51.69% -5.60%
+ Complexity 18532 16533 -1999
============================================
Files 1944 1944
Lines 106132 106157 +25
Branches 13118 13124 +6
============================================
- Hits 60797 54877 -5920
- Misses 39612 45373 +5761
- Partials 5723 5907 +184
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.23% <0.00%> (-14.20%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.25% <0.00%> (-0.26%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
... and [535 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945073382
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 55.54%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 55.54% -1.75%
+ Complexity 18532 17919 -613
============================================
Files 1944 1944
Lines 106132 106157 +25
Branches 13118 13124 +6
============================================
- Hits 60797 58960 -1837
- Misses 39612 41408 +1796
- Partials 5723 5789 +66
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.53% <0.00%> (-3.90%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
... and [233 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944740253
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 02979148bf26522bbb63b4128e06a516684e37cf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148)
* 1f1bb42055b7f176cb4d4d2bd614de433d99740a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949253385
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 57.27%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 57.27% -0.02%
- Complexity 18532 18536 +4
============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
============================================
+ Hits 60797 60801 +4
- Misses 39612 39640 +28
Partials 5723 5723
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.42% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
... and [10 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2850644442
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java:
##########
@@ -403,16 +405,45 @@ public LookupRuntimeProvider getLookupRuntimeProvider(LookupContext context) {
Duration duration = conf.get(LOOKUP_JOIN_CACHE_TTL);
boolean asyncEnabled = conf.get(LOOKUP_ASYNC);
int asyncThreadNumber = conf.get(LOOKUP_ASYNC_THREAD_NUMBER);
+ Option<PartitionPruners.PartitionPruner> lookupPruner = buildLookupPartitionPruner();
return LookupRuntimeProviderFactory.create(
new HoodieLookupFunction(
- new HoodieLookupTableReader(this::getBatchInputFormat, conf),
+ new HoodieLookupTableReader(
+ lookupPruner.isPresent()
+ ? () -> getBatchInputFormatWithPruner(lookupPruner.get())
+ : this::getBatchInputFormat,
+ conf),
(RowType) getProducedDataType().notNull().getLogicalType(),
getLookupKeys(context.getKeys()),
duration,
conf
), asyncEnabled, asyncThreadNumber);
}
+ private Option<PartitionPruners.PartitionPruner> buildLookupPartitionPruner() {
+ if (!conf.contains(LOOKUP_PARTITIONS) || !isPartitioned()) {
+ return Option.empty();
+ }
+ List<String> partitionPaths = PartitionPathParser.parseLookupPartitionPaths(
+ conf.get(LOOKUP_PARTITIONS), this.partitionKeys, conf.get(FlinkOptions.HIVE_STYLE_PARTITIONING));
+ if (partitionPaths.isEmpty()) {
+ return Option.empty();
+ }
+ return Option.of(PartitionPruners.builder()
+ .candidatePartitions(partitionPaths)
+ .build());
+ }
+
+ private InputFormat<RowData, ?> getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner lookupPruner) {
+ PartitionPruners.PartitionPruner saved = this.partitionPruner;
+ this.partitionPruner = lookupPruner;
+ try {
+ return getBatchInputFormat();
+ } finally {
+ this.partitionPruner = saved;
Review Comment:
you are right, the original code looks simpler, we can revert to the old impl as long as we can ensure the file index is not reused for other scan sources. maybe we just nullify the file index manually here to make it safe.
And if we can use SQL filter push down to specifiy the partitions of dim table, there is no need to make any changes to the `HoodieTableSource`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-4088661486
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 57.28%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cbe77a8`](https://app.codecov.io/gh/apache/hudi/commit/cbe77a8b82bd7b752a9364f75fa32e409b7539eb?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 62 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=========================================
Coverage 57.28% 57.28%
- Complexity 18532 18555 +23
=========================================
Files 1944 1945 +1
Lines 106132 106226 +94
Branches 13118 13135 +17
=========================================
+ Hits 60797 60854 +57
- Misses 39612 39648 +36
- Partials 5723 5724 +1
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.41% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.51% <0.00%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
... and [16 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3959417375
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
}, {
"hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
"triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"triggerType" : "PUSH"
}, {
"hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
"triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"triggerType" : "PUSH"
}, {
"hash" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12293",
"triggerID" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cbe77a8b82bd7b752a9364f75fa32e409b7539eb Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12293)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2850652704
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java:
##########
@@ -1325,6 +1325,15 @@ public class FlinkOptions extends HoodieConfig {
+ "Each task manager will create a unique subdirectory under this path. "
+ "The directory is cleaned up when the lookup function is closed.");
+ public static final ConfigOption<String> LOOKUP_PARTITIONS =
+ key("lookup.partitions")
Review Comment:
it looks like dim table already supports filer push down through SQL: https://cwiki.apache.org/confluence/display/FLINK/FLIP-324%3A+Introduce+Runtime+Filter+for+Flink+Batch+Jobs, we can utilize the SQL filters instead of the explicit options.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944753261
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 02979148bf26522bbb63b4128e06a516684e37cf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148)
* 1f1bb42055b7f176cb4d4d2bd614de433d99740a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3948912987
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
}, {
"hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219)
* 0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949582942
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
}, {
"hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
"triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"triggerType" : "PUSH"
}, {
"hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
"triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251)
* f1ea3e2b59bfa373ccca2abf12004acb221e8639 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945031725
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 55.44%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 55.44% -1.85%
+ Complexity 18532 17882 -650
============================================
Files 1944 1944
Lines 106132 106157 +25
Branches 13118 13124 +6
============================================
- Hits 60797 58858 -1939
- Misses 39612 41496 +1884
- Partials 5723 5803 +80
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.37% <0.00%> (-4.06%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
... and [254 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2850644442
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java:
##########
@@ -403,16 +405,45 @@ public LookupRuntimeProvider getLookupRuntimeProvider(LookupContext context) {
Duration duration = conf.get(LOOKUP_JOIN_CACHE_TTL);
boolean asyncEnabled = conf.get(LOOKUP_ASYNC);
int asyncThreadNumber = conf.get(LOOKUP_ASYNC_THREAD_NUMBER);
+ Option<PartitionPruners.PartitionPruner> lookupPruner = buildLookupPartitionPruner();
return LookupRuntimeProviderFactory.create(
new HoodieLookupFunction(
- new HoodieLookupTableReader(this::getBatchInputFormat, conf),
+ new HoodieLookupTableReader(
+ lookupPruner.isPresent()
+ ? () -> getBatchInputFormatWithPruner(lookupPruner.get())
+ : this::getBatchInputFormat,
+ conf),
(RowType) getProducedDataType().notNull().getLogicalType(),
getLookupKeys(context.getKeys()),
duration,
conf
), asyncEnabled, asyncThreadNumber);
}
+ private Option<PartitionPruners.PartitionPruner> buildLookupPartitionPruner() {
+ if (!conf.contains(LOOKUP_PARTITIONS) || !isPartitioned()) {
+ return Option.empty();
+ }
+ List<String> partitionPaths = PartitionPathParser.parseLookupPartitionPaths(
+ conf.get(LOOKUP_PARTITIONS), this.partitionKeys, conf.get(FlinkOptions.HIVE_STYLE_PARTITIONING));
+ if (partitionPaths.isEmpty()) {
+ return Option.empty();
+ }
+ return Option.of(PartitionPruners.builder()
+ .candidatePartitions(partitionPaths)
+ .build());
+ }
+
+ private InputFormat<RowData, ?> getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner lookupPruner) {
+ PartitionPruners.PartitionPruner saved = this.partitionPruner;
+ this.partitionPruner = lookupPruner;
+ try {
+ return getBatchInputFormat();
+ } finally {
+ this.partitionPruner = saved;
Review Comment:
you are right, the original code looks simpler, we can revert to the old impl as long as we can ensure the file index is not reused for other scan sources. maybe we just nullify the file index manually here to make it safe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2840897721
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java:
##########
@@ -403,16 +405,45 @@ public LookupRuntimeProvider getLookupRuntimeProvider(LookupContext context) {
Duration duration = conf.get(LOOKUP_JOIN_CACHE_TTL);
boolean asyncEnabled = conf.get(LOOKUP_ASYNC);
int asyncThreadNumber = conf.get(LOOKUP_ASYNC_THREAD_NUMBER);
+ Option<PartitionPruners.PartitionPruner> lookupPruner = buildLookupPartitionPruner();
return LookupRuntimeProviderFactory.create(
new HoodieLookupFunction(
- new HoodieLookupTableReader(this::getBatchInputFormat, conf),
+ new HoodieLookupTableReader(
+ lookupPruner.isPresent()
+ ? () -> getBatchInputFormatWithPruner(lookupPruner.get())
+ : this::getBatchInputFormat,
+ conf),
(RowType) getProducedDataType().notNull().getLogicalType(),
getLookupKeys(context.getKeys()),
duration,
conf
), asyncEnabled, asyncThreadNumber);
}
+ private Option<PartitionPruners.PartitionPruner> buildLookupPartitionPruner() {
+ if (!conf.contains(LOOKUP_PARTITIONS) || !isPartitioned()) {
+ return Option.empty();
+ }
+ List<String> partitionPaths = PartitionPathParser.parseLookupPartitionPaths(
+ conf.get(LOOKUP_PARTITIONS), this.partitionKeys, conf.get(FlinkOptions.HIVE_STYLE_PARTITIONING));
+ if (partitionPaths.isEmpty()) {
+ return Option.empty();
+ }
+ return Option.of(PartitionPruners.builder()
+ .candidatePartitions(partitionPaths)
+ .build());
+ }
+
+ private InputFormat<RowData, ?> getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner lookupPruner) {
+ PartitionPruners.PartitionPruner saved = this.partitionPruner;
+ this.partitionPruner = lookupPruner;
+ try {
+ return getBatchInputFormat();
+ } finally {
+ this.partitionPruner = saved;
Review Comment:
@danny0405 Ok, i've refactored this code (in separate commit). But it was not enough to just pass around PartitionPruner into getBatchInputFormat, because it is used deeper. (in `FileIndexReader`) I think, now it's not less error-prone, but more complicated.
Review it please, if it's ok for you, approve it, then I'll actualize PR description and merge.
If during review you decide that the first version was better - let me know, i'll drop the second commit and re-request review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945026880
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 51.81%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
>
> <details><summary>HEAD has 7 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (5b7100c) |
>|------|------|------|
>|spark-scala-tests|6|5|
>|spark-java-tests|9|3|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 51.81% -5.47%
+ Complexity 18532 16561 -1971
============================================
Files 1944 1944
Lines 106132 106157 +25
Branches 13118 13124 +6
============================================
- Hits 60797 55009 -5788
- Misses 39612 45248 +5636
- Partials 5723 5900 +177
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.23% <0.00%> (-14.20%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.42% <0.00%> (-0.09%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
... and [506 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945039291
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 55.47%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 55.47% -1.82%
+ Complexity 18532 17890 -642
============================================
Files 1944 1944
Lines 106132 106157 +25
Branches 13118 13124 +6
============================================
- Hits 60797 58887 -1910
- Misses 39612 41469 +1857
- Partials 5723 5801 +78
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.42% <0.00%> (-4.01%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
... and [251 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945049304
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 02979148bf26522bbb63b4128e06a516684e37cf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148)
* 1f1bb42055b7f176cb4d4d2bd614de433d99740a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215)
* 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949161545
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 51.71%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
>
> <details><summary>HEAD has 8 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
>|------|------|------|
>|spark-scala-tests|6|4|
>|spark-java-tests|9|3|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 51.71% -5.58%
+ Complexity 18532 16532 -2000
============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
============================================
- Hits 60797 54898 -5899
- Misses 39612 45353 +5741
- Partials 5723 5913 +190
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.22% <0.00%> (-14.20%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.27% <0.00%> (-0.24%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
... and [527 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949171599
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 55.30%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 55.30% -1.98%
+ Complexity 18532 17856 -676
============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
============================================
- Hits 60797 58718 -2079
- Misses 39612 41662 +2050
- Partials 5723 5784 +61
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.20% <0.00%> (-4.23%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
... and [269 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944835871
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.39%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
>
> <details><summary>HEAD has 15 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (5b7100c) |
>|------|------|------|
>|spark-scala-tests|6|0|
>|spark-java-tests|9|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=============================================
- Coverage 57.28% 45.39% -11.89%
+ Complexity 18532 8370 -10162
=============================================
Files 1944 1177 -767
Lines 106132 60264 -45868
Branches 13118 6420 -6698
=============================================
- Hits 60797 27357 -33440
+ Misses 39612 29933 -9679
+ Partials 5723 2974 -2749
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `17.20% <0.00%> (-9.27%)` | :arrow_down: |
... and [1381 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2840897721
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java:
##########
@@ -403,16 +405,45 @@ public LookupRuntimeProvider getLookupRuntimeProvider(LookupContext context) {
Duration duration = conf.get(LOOKUP_JOIN_CACHE_TTL);
boolean asyncEnabled = conf.get(LOOKUP_ASYNC);
int asyncThreadNumber = conf.get(LOOKUP_ASYNC_THREAD_NUMBER);
+ Option<PartitionPruners.PartitionPruner> lookupPruner = buildLookupPartitionPruner();
return LookupRuntimeProviderFactory.create(
new HoodieLookupFunction(
- new HoodieLookupTableReader(this::getBatchInputFormat, conf),
+ new HoodieLookupTableReader(
+ lookupPruner.isPresent()
+ ? () -> getBatchInputFormatWithPruner(lookupPruner.get())
+ : this::getBatchInputFormat,
+ conf),
(RowType) getProducedDataType().notNull().getLogicalType(),
getLookupKeys(context.getKeys()),
duration,
conf
), asyncEnabled, asyncThreadNumber);
}
+ private Option<PartitionPruners.PartitionPruner> buildLookupPartitionPruner() {
+ if (!conf.contains(LOOKUP_PARTITIONS) || !isPartitioned()) {
+ return Option.empty();
+ }
+ List<String> partitionPaths = PartitionPathParser.parseLookupPartitionPaths(
+ conf.get(LOOKUP_PARTITIONS), this.partitionKeys, conf.get(FlinkOptions.HIVE_STYLE_PARTITIONING));
+ if (partitionPaths.isEmpty()) {
+ return Option.empty();
+ }
+ return Option.of(PartitionPruners.builder()
+ .candidatePartitions(partitionPaths)
+ .build());
+ }
+
+ private InputFormat<RowData, ?> getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner lookupPruner) {
+ PartitionPruners.PartitionPruner saved = this.partitionPruner;
+ this.partitionPruner = lookupPruner;
+ try {
+ return getBatchInputFormat();
+ } finally {
+ this.partitionPruner = saved;
Review Comment:
@danny0405 Ok, i've refactored this code (in separate commit). Agree with you about error-prone, but it was not enough to just pass around PartitionPruner into getBatchInputFormat, because it is used deeper (in `FileIndexReader`). I think, now it becomes not less error-prone, but more complicated.
Review it please, if it's ok for you, approve it, then I'll actualize PR description and merge.
If during review you decide that the first version was better - let me know, i'll drop the second commit and re-request review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944767076
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 02979148bf26522bbb63b4128e06a516684e37cf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148)
* 1f1bb42055b7f176cb4d4d2bd614de433d99740a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215)
* 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3950145810
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
}, {
"hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
"triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
"triggerType" : "PUSH"
}, {
"hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
"triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
"triggerType" : "PUSH"
}, {
"hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
"triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
"triggerType" : "PUSH"
}, {
"hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
"triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f1ea3e2b59bfa373ccca2abf12004acb221e8639 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949165258
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 51.86%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
>
> <details><summary>HEAD has 6 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
>|------|------|------|
>|spark-java-tests|9|3|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 51.86% -5.43%
+ Complexity 18532 16577 -1955
============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
============================================
- Hits 60797 55057 -5740
- Misses 39612 45201 +5589
- Partials 5723 5906 +183
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.22% <0.00%> (-14.20%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
... and [501 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949138791
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 43.53%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
>
> <details><summary>HEAD has 9 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
>|------|------|------|
>|spark-scala-tests|6|3|
>|spark-java-tests|9|3|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=============================================
- Coverage 57.28% 43.53% -13.76%
+ Complexity 18532 13836 -4696
=============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
=============================================
- Hits 60797 46216 -14581
- Misses 39612 54651 +15039
+ Partials 5723 5297 -426
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.22% <0.00%> (-14.20%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
... and [784 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932500079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8d6d98805ddabfa1da4ff4df6304984b37ec6982 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147)
* 02979148bf26522bbb63b4128e06a516684e37cf Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932208141
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
"triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
"triggerType" : "PUSH"
}, {
"hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8d6d98805ddabfa1da4ff4df6304984b37ec6982 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147)
* 02979148bf26522bbb63b4128e06a516684e37cf UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2838975681
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java:
##########
@@ -403,16 +405,45 @@ public LookupRuntimeProvider getLookupRuntimeProvider(LookupContext context) {
Duration duration = conf.get(LOOKUP_JOIN_CACHE_TTL);
boolean asyncEnabled = conf.get(LOOKUP_ASYNC);
int asyncThreadNumber = conf.get(LOOKUP_ASYNC_THREAD_NUMBER);
+ Option<PartitionPruners.PartitionPruner> lookupPruner = buildLookupPartitionPruner();
return LookupRuntimeProviderFactory.create(
new HoodieLookupFunction(
- new HoodieLookupTableReader(this::getBatchInputFormat, conf),
+ new HoodieLookupTableReader(
+ lookupPruner.isPresent()
+ ? () -> getBatchInputFormatWithPruner(lookupPruner.get())
+ : this::getBatchInputFormat,
+ conf),
(RowType) getProducedDataType().notNull().getLogicalType(),
getLookupKeys(context.getKeys()),
duration,
conf
), asyncEnabled, asyncThreadNumber);
}
+ private Option<PartitionPruners.PartitionPruner> buildLookupPartitionPruner() {
+ if (!conf.contains(LOOKUP_PARTITIONS) || !isPartitioned()) {
+ return Option.empty();
+ }
+ List<String> partitionPaths = PartitionPathParser.parseLookupPartitionPaths(
+ conf.get(LOOKUP_PARTITIONS), this.partitionKeys, conf.get(FlinkOptions.HIVE_STYLE_PARTITIONING));
+ if (partitionPaths.isEmpty()) {
+ return Option.empty();
+ }
+ return Option.of(PartitionPruners.builder()
+ .candidatePartitions(partitionPaths)
+ .build());
+ }
+
+ private InputFormat<RowData, ?> getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner lookupPruner) {
+ PartitionPruners.PartitionPruner saved = this.partitionPruner;
+ this.partitionPruner = lookupPruner;
+ try {
+ return getBatchInputFormat();
+ } finally {
+ this.partitionPruner = saved;
Review Comment:
this is error prone, just pass around the `PartitionPruner` as a param into `getBatchInputFormat` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945028225
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 51.86%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
>
> <details><summary>HEAD has 6 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (5b7100c) |
>|------|------|------|
>|spark-java-tests|9|3|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 51.86% -5.43%
+ Complexity 18532 16578 -1954
============================================
Files 1944 1944
Lines 106132 106157 +25
Branches 13118 13124 +6
============================================
- Hits 60797 55057 -5740
- Misses 39612 45198 +5586
- Partials 5723 5902 +179
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.23% <0.00%> (-14.20%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
... and [498 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949175801
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 55.50%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
============================================
- Coverage 57.28% 55.50% -1.78%
+ Complexity 18532 17911 -621
============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
============================================
- Hits 60797 58930 -1867
- Misses 39612 41439 +1827
- Partials 5723 5795 +72
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.47% <0.00%> (-3.95%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
... and [243 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944971116
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 43.53%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
>
> <details><summary>HEAD has 9 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (5b7100c) |
>|------|------|------|
>|spark-scala-tests|6|3|
>|spark-java-tests|9|3|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=============================================
- Coverage 57.28% 43.53% -13.75%
+ Complexity 18532 13837 -4695
=============================================
Files 1944 1944
Lines 106132 106157 +25
Branches 13118 13124 +6
=============================================
- Hits 60797 46217 -14580
- Misses 39612 54646 +15034
+ Partials 5723 5294 -429
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.23% <0.00%> (-14.20%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `17.20% <0.00%> (-9.27%)` | :arrow_down: |
... and [782 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944957018
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 39.70%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
>
> <details><summary>HEAD has 12 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (5b7100c) |
>|------|------|------|
>|spark-scala-tests|6|3|
>|spark-java-tests|9|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=============================================
- Coverage 57.28% 39.70% -17.58%
+ Complexity 18532 12574 -5958
=============================================
Files 1944 1944
Lines 106132 106157 +25
Branches 13118 13124 +6
=============================================
- Hits 60797 42153 -18644
- Misses 39612 58945 +19333
+ Partials 5723 5059 -664
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `17.20% <0.00%> (-9.27%)` | :arrow_down: |
... and [856 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949124126
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 43.49%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
>
> <details><summary>HEAD has 10 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
>|------|------|------|
>|spark-scala-tests|6|3|
>|spark-java-tests|9|2|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=============================================
- Coverage 57.28% 43.49% -13.80%
+ Complexity 18532 13823 -4709
=============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
=============================================
- Hits 60797 46171 -14626
- Misses 39612 54691 +15079
+ Partials 5723 5302 -421
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.67% <0.00%> (-13.76%)` | :arrow_down: |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
... and [786 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949098386
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 39.70%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
>
> <details><summary>HEAD has 12 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
>|------|------|------|
>|spark-scala-tests|6|3|
>|spark-java-tests|9|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=============================================
- Coverage 57.28% 39.70% -17.58%
+ Complexity 18532 12575 -5957
=============================================
Files 1944 1944
Lines 106132 106164 +32
Branches 13118 13126 +8
=============================================
- Hits 60797 42155 -18642
- Misses 39612 58949 +19337
+ Partials 5723 5060 -663
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
... and [858 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2852815343
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java:
##########
@@ -1325,6 +1325,15 @@ public class FlinkOptions extends HoodieConfig {
+ "Each task manager will create a unique subdirectory under this path. "
+ "The directory is cleaned up when the lookup function is closed.");
+ public static final ConfigOption<String> LOOKUP_PARTITIONS =
+ key("lookup.partitions")
Review Comment:
Sorry, i don't understand your point. Could you please explain your idea?
Why using SQL filters are better than pruning partitions directly?
Or you mean, that users prefer to write SQL with "... where dim.partitionField='par1' or dim.partitionField='par2'"? then this whole PR is not needed at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3948973952
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.40%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
>
> <details><summary>HEAD has 15 uploads less than BASE</summary>
>
>| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
>|------|------|------|
>|spark-scala-tests|6|0|
>|spark-java-tests|9|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18229 +/- ##
=============================================
- Coverage 57.28% 45.40% -11.89%
+ Complexity 18532 8372 -10160
=============================================
Files 1944 1177 -767
Lines 106132 60266 -45866
Branches 13118 6421 -6697
=============================================
- Hits 60797 27362 -33435
+ Misses 39612 29930 -9682
+ Partials 5723 2974 -2749
```
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
| [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
... and [1383 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2844245487
##########
hudi-common/src/main/java/org/apache/hudi/common/table/PartitionPathParser.java:
##########
@@ -164,4 +171,53 @@ private static Object inferDateValue(
private static boolean isTimeBasedType(HoodieSchemaType type) {
return type == HoodieSchemaType.DATE || type == HoodieSchemaType.TIMESTAMP || type == HoodieSchemaType.TIME;
}
+
+ /**
+ * Parses the {@code lookup.partitions} option value into a list of Hudi partition paths.
+ *
+ * <p>The spec format is {@code key1=val1,key2=val2} per partition, with multiple partitions
+ * separated by {@code ;}. Example: {@code "dt=2024-01-01,region=us;dt=2024-01-02,region=eu"}.
+ *
+ * @param spec the raw option value
+ * @param partitionKeys ordered list of partition key names as defined in the table schema
+ * @param hiveStyle whether the table uses Hive-style partitioning ({@code key=value} directories)
+ * @return list of partition paths in the format used by Hudi's file index
+ * @throws IllegalArgumentException if any key in the spec is not a valid partition key,
+ * or if a key-value pair does not follow {@code key=value} format
+ */
+ public static List<String> parseLookupPartitionPaths(String spec, List<String> partitionKeys, boolean hiveStyle) {
Review Comment:
ok, i'll do it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org