You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "wombatu-kun (via GitHub)" <gi...@apache.org> on 2026/02/20 07:33:59 UTC

[PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

wombatu-kun opened a new pull request, #18229:
URL: https://github.com/apache/hudi/pull/18229

   ### Describe the issue this Pull Request addresses
   
   When using Hudi as a lookup join dimension table in Flink, the entire table is loaded into the cache on each reload. For large dimension tables that are partitioned by a slowly-changing attribute (e.g., date, region), this means reading and caching data from all partitions — including historical ones that are irrelevant to the running job. This can cause excessive memory pressure and slow cache warm-up times.  
   This PR introduces a new option lookup.partitions that allows users to declare which partitions should be loaded into the lookup join cache, so that only the relevant slice of the dimension table is materialized in memory.
   
   ### Summary and Changelog
   
   Users can now limit the lookup join cache to a subset of partitions by specifying the `lookup.partitions` option via a query hint. This is analogous to the existing lookup.async option that controls the lookup execution mode.
   
   Partitions are expressed as comma-separated `key=value pairs` (one partition per entry), with multiple partitions separated by ;:
   
   hudi-flink-datasource/hudi-flink — FlinkOptions.java
   Added LOOKUP_PARTITIONS (lookup.partitions) config option of type String with no default value.  
   
   hudi-flink-datasource/hudi-flink — HoodieTableSource.java
   Modified getLookupRuntimeProvider() to build an optional partition pruner from lookup.partitions and pass it to the HoodieLookupTableReader supplier when the option is set.
   Added buildLookupPartitionPruner(): reads lookup.partitions, delegates path parsing to PartitionPathParser, and wraps the result in a PartitionPruners.StaticPartitionPruner via the existing PartitionPruners.Builder API.
   Added getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner): temporarily substitutes this.partitionPruner with the lookup-specific pruner before calling getBatchInputFormat(), then restores the original value. This allows the lookup path to reuse all existing batch input format logic (COW, MOR, read-optimized) without duplication.  
   
   hudi-common — PartitionPathParser.java
   Added parseLookupPartitionPaths(String spec, List<String> partitionKeys, boolean hiveStyle): parses the lookup.partitions spec into a list of Hudi partition paths.
   Validates that every key in the spec belongs to the table's declared partition key set; throws IllegalArgumentException with an informative message (including the list of valid keys) on unknown keys.
   Validates that every token follows key=value format; throws IllegalArgumentException on bare values without a key.
   Constructs paths in partitionKeys order regardless of the order keys appear in the spec, ensuring correct path generation for both Hive-style (key=value/key=value) and plain-value (value/value) layouts.
   
   ### Impact
   
   When lookup.partitions is not configured — the feature is fully opt-in and backward-compatible. When configured, only the specified partitions are read during cache population; rows whose lookup keys resolve to an excluded partition will not match any cache entry and will produce null values on the dimension side of a LEFT lookup join.  
   Cache warm-up time and memory footprint are reduced proportionally to the fraction of partitions excluded.
   
   ### Risk Level
   
   none
   
   ### Documentation Update
   
   Documentation update is needed: to add new FlinkOption `lookup.partitions`.
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Enough context is provided in the sections above
   - [ ] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945591267

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949164439

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 51.81%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
   > 
   > <details><summary>HEAD has 7 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
   >|------|------|------|
   >|spark-scala-tests|6|5|
   >|spark-java-tests|9|3|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   51.81%   -5.47%     
   + Complexity    18532    16563    -1969     
   ============================================
     Files          1944     1944              
     Lines        106132   106164      +32     
     Branches      13118    13126       +8     
   ============================================
   - Hits          60797    55013    -5784     
   - Misses        39612    45248    +5636     
   - Partials       5723     5903     +180     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.22% <0.00%> (-14.20%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.42% <0.00%> (-0.09%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
   
   ... and [508 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944959660

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 39.70%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
   > 
   > <details><summary>HEAD has 12 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (5b7100c) |
   >|------|------|------|
   >|spark-scala-tests|6|3|
   >|spark-java-tests|9|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #18229       +/-   ##
   =============================================
   - Coverage     57.28%   39.70%   -17.58%     
   + Complexity    18532    12574     -5958     
   =============================================
     Files          1944     1944               
     Lines        106132   106157       +25     
     Branches      13118    13124        +6     
   =============================================
   - Hits          60797    42153    -18644     
   - Misses        39612    58945    +19333     
   + Partials       5723     5059      -664     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `17.20% <0.00%> (-9.27%)` | :arrow_down: |
   
   ... and [856 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949099690

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 39.70%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
   > 
   > <details><summary>HEAD has 12 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
   >|------|------|------|
   >|spark-scala-tests|6|3|
   >|spark-java-tests|9|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #18229       +/-   ##
   =============================================
   - Coverage     57.28%   39.70%   -17.58%     
   + Complexity    18532    12575     -5957     
   =============================================
     Files          1944     1944               
     Lines        106132   106164       +32     
     Branches      13118    13126        +8     
   =============================================
   - Hits          60797    42155    -18642     
   - Misses        39612    58949    +19337     
   + Partials       5723     5060      -663     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
   
   ... and [858 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949114686

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 43.42%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
   > 
   > <details><summary>HEAD has 11 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
   >|------|------|------|
   >|spark-scala-tests|6|3|
   >|spark-java-tests|9|1|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #18229       +/-   ##
   =============================================
   - Coverage     57.28%   43.42%   -13.86%     
   + Complexity    18532    13807     -4725     
   =============================================
     Files          1944     1944               
     Lines        106132   106164       +32     
     Branches      13118    13126        +8     
   =============================================
   - Hits          60797    46104    -14693     
   - Misses        39612    54749    +15137     
   + Partials       5723     5311      -412     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.08% <0.00%> (-13.35%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
   
   ... and [787 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949177596

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 55.53%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   55.53%   -1.75%     
   + Complexity    18532    17918     -614     
   ============================================
     Files          1944     1944              
     Lines        106132   106164      +32     
     Branches      13118    13126       +8     
   ============================================
   - Hits          60797    58960    -1837     
   - Misses        39612    41411    +1799     
   - Partials       5723     5793      +70     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.53% <0.00%> (-3.90%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
   
   ... and [235 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932188080

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8d6d98805ddabfa1da4ff4df6304984b37ec6982 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932194714

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8d6d98805ddabfa1da4ff4df6304984b37ec6982 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945199225

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 1f1bb42055b7f176cb4d4d2bd614de433d99740a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215) 
   * 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932438384

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8d6d98805ddabfa1da4ff4df6304984b37ec6982 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147) 
   * 02979148bf26522bbb63b4128e06a516684e37cf Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932789247

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 02979148bf26522bbb63b4128e06a516684e37cf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945129845

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 57.27%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   57.27%   -0.02%     
   - Complexity    18532    18536       +4     
   ============================================
     Files          1944     1944              
     Lines        106132   106157      +25     
     Branches      13118    13124       +6     
   ============================================
   + Hits          60797    60798       +1     
   - Misses        39612    39639      +27     
   + Partials       5723     5720       -3     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.42% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
   
   ... and [10 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "cshuo (via GitHub)" <gi...@apache.org>.
cshuo commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2844215651


##########
hudi-common/src/main/java/org/apache/hudi/common/table/PartitionPathParser.java:
##########
@@ -164,4 +171,53 @@ private static Object inferDateValue(
   private static boolean isTimeBasedType(HoodieSchemaType type) {
     return type == HoodieSchemaType.DATE || type == HoodieSchemaType.TIMESTAMP || type == HoodieSchemaType.TIME;
   }
+
+  /**
+   * Parses the {@code lookup.partitions} option value into a list of Hudi partition paths.
+   *
+   * <p>The spec format is {@code key1=val1,key2=val2} per partition, with multiple partitions
+   * separated by {@code ;}. Example: {@code "dt=2024-01-01,region=us;dt=2024-01-02,region=eu"}.
+   *
+   * @param spec         the raw option value
+   * @param partitionKeys ordered list of partition key names as defined in the table schema
+   * @param hiveStyle    whether the table uses Hive-style partitioning ({@code key=value} directories)
+   * @return list of partition paths in the format used by Hudi's file index
+   * @throws IllegalArgumentException if any key in the spec is not a valid partition key,
+   *                                  or if a key-value pair does not follow {@code key=value} format
+   */
+  public static List<String> parseLookupPartitionPaths(String spec, List<String> partitionKeys, boolean hiveStyle) {

Review Comment:
   lookup.partitions currently allows partial key sets for multi-key partitioned tables (e.g. only year=2024 when partition keys are year,month).
   For lookup pruning, this is risky because StaticPartitionPruner does exact partition-path matching. A partial path can silently fail to match real partition directories (e.g. year=2024/month=01), resulting in empty lookup cache for expected rows.
   Could we enforce that each partition spec includes all declared partition keys (fail fast with IllegalArgumentException if any key is missing)?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949242641

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 57.26%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   57.26%   -0.03%     
     Complexity    18532    18532              
   ============================================
     Files          1944     1944              
     Lines        106132   106164      +32     
     Branches      13118    13126       +8     
   ============================================
   - Hits          60797    60791       -6     
   - Misses        39612    39646      +34     
   - Partials       5723     5727       +4     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.34% <0.00%> (-0.08%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
   
   ... and [12 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3958948075

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
       "triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
       "triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12293",
       "triggerID" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f1ea3e2b59bfa373ccca2abf12004acb221e8639 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258) 
   * cbe77a8b82bd7b752a9364f75fa32e409b7539eb Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12293) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3958933488

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
       "triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
       "triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f1ea3e2b59bfa373ccca2abf12004acb221e8639 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258) 
   * cbe77a8b82bd7b752a9364f75fa32e409b7539eb UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945100796

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 57.26%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   57.26%   -0.02%     
   - Complexity    18532    18534       +2     
   ============================================
     Files          1944     1944              
     Lines        106132   106157      +25     
     Branches      13118    13124       +6     
   ============================================
   - Hits          60797    60792       -5     
   - Misses        39612    39643      +31     
   + Partials       5723     5722       -1     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.35% <0.00%> (-0.08%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
   
   ... and [10 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3948928754

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
       "triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219) 
   * 0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251) 
   * f1ea3e2b59bfa373ccca2abf12004acb221e8639 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3948919348

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
       "triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219) 
   * 0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949194383

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
       "triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
       "triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219) 
   * 0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251) 
   * f1ea3e2b59bfa373ccca2abf12004acb221e8639 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949247478

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 57.26%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   57.26%   -0.03%     
   - Complexity    18532    18534       +2     
   ============================================
     Files          1944     1944              
     Lines        106132   106164      +32     
     Branches      13118    13126       +8     
   ============================================
   - Hits          60797    60793       -4     
   - Misses        39612    39645      +33     
   - Partials       5723     5726       +3     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.38% <0.00%> (-0.05%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
   
   ... and [13 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944941505

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 40.51%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
   > 
   > <details><summary>HEAD has 14 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (5b7100c) |
   >|------|------|------|
   >|spark-scala-tests|6|1|
   >|spark-java-tests|9|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #18229       +/-   ##
   =============================================
   - Coverage     57.28%   40.51%   -16.77%     
   + Complexity    18532    12237     -6295     
   =============================================
     Files          1944     1910       -34     
     Lines        106132   101406     -4726     
     Branches      13118    12026     -1092     
   =============================================
   - Hits          60797    41087    -19710     
   - Misses        39612    55498    +15886     
   + Partials       5723     4821      -902     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.64% <0.00%> (-11.87%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `17.20% <0.00%> (-9.27%)` | :arrow_down: |
   
   ... and [884 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945022855

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 51.69%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
   > 
   > <details><summary>HEAD has 8 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (5b7100c) |
   >|------|------|------|
   >|spark-scala-tests|6|4|
   >|spark-java-tests|9|3|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   51.69%   -5.60%     
   + Complexity    18532    16533    -1999     
   ============================================
     Files          1944     1944              
     Lines        106132   106157      +25     
     Branches      13118    13124       +6     
   ============================================
   - Hits          60797    54877    -5920     
   - Misses        39612    45373    +5761     
   - Partials       5723     5907     +184     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.23% <0.00%> (-14.20%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.25% <0.00%> (-0.26%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
   
   ... and [535 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945073382

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 55.54%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   55.54%   -1.75%     
   + Complexity    18532    17919     -613     
   ============================================
     Files          1944     1944              
     Lines        106132   106157      +25     
     Branches      13118    13124       +6     
   ============================================
   - Hits          60797    58960    -1837     
   - Misses        39612    41408    +1796     
   - Partials       5723     5789      +66     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.53% <0.00%> (-3.90%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
   
   ... and [233 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944740253

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 02979148bf26522bbb63b4128e06a516684e37cf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148) 
   * 1f1bb42055b7f176cb4d4d2bd614de433d99740a UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949253385

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 57.27%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   57.27%   -0.02%     
   - Complexity    18532    18536       +4     
   ============================================
     Files          1944     1944              
     Lines        106132   106164      +32     
     Branches      13118    13126       +8     
   ============================================
   + Hits          60797    60801       +4     
   - Misses        39612    39640      +28     
     Partials       5723     5723              
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.42% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
   
   ... and [10 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2850644442


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java:
##########
@@ -403,16 +405,45 @@ public LookupRuntimeProvider getLookupRuntimeProvider(LookupContext context) {
     Duration duration = conf.get(LOOKUP_JOIN_CACHE_TTL);
     boolean asyncEnabled = conf.get(LOOKUP_ASYNC);
     int asyncThreadNumber = conf.get(LOOKUP_ASYNC_THREAD_NUMBER);
+    Option<PartitionPruners.PartitionPruner> lookupPruner = buildLookupPartitionPruner();
     return LookupRuntimeProviderFactory.create(
         new HoodieLookupFunction(
-            new HoodieLookupTableReader(this::getBatchInputFormat, conf),
+            new HoodieLookupTableReader(
+                lookupPruner.isPresent()
+                    ? () -> getBatchInputFormatWithPruner(lookupPruner.get())
+                    : this::getBatchInputFormat,
+                conf),
             (RowType) getProducedDataType().notNull().getLogicalType(),
             getLookupKeys(context.getKeys()),
             duration,
             conf
         ), asyncEnabled, asyncThreadNumber);
   }
 
+  private Option<PartitionPruners.PartitionPruner> buildLookupPartitionPruner() {
+    if (!conf.contains(LOOKUP_PARTITIONS) || !isPartitioned()) {
+      return Option.empty();
+    }
+    List<String> partitionPaths = PartitionPathParser.parseLookupPartitionPaths(
+        conf.get(LOOKUP_PARTITIONS), this.partitionKeys, conf.get(FlinkOptions.HIVE_STYLE_PARTITIONING));
+    if (partitionPaths.isEmpty()) {
+      return Option.empty();
+    }
+    return Option.of(PartitionPruners.builder()
+        .candidatePartitions(partitionPaths)
+        .build());
+  }
+
+  private InputFormat<RowData, ?> getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner lookupPruner) {
+    PartitionPruners.PartitionPruner saved = this.partitionPruner;
+    this.partitionPruner = lookupPruner;
+    try {
+      return getBatchInputFormat();
+    } finally {
+      this.partitionPruner = saved;

Review Comment:
   you are right, the original code looks simpler, we can revert to the old impl as long as we can ensure the file index is not reused for other scan sources. maybe we just nullify the file index manually here to make it safe.
   
   And if we can use SQL filter push down to specifiy the partitions of dim table, there is no need to make any changes to the `HoodieTableSource`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-4088661486

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 57.28%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cbe77a8`](https://app.codecov.io/gh/apache/hudi/commit/cbe77a8b82bd7b752a9364f75fa32e409b7539eb?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 62 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@            Coverage Diff            @@
   ##             master   #18229   +/-   ##
   =========================================
     Coverage     57.28%   57.28%           
   - Complexity    18532    18555   +23     
   =========================================
     Files          1944     1945    +1     
     Lines        106132   106226   +94     
     Branches      13118    13135   +17     
   =========================================
   + Hits          60797    60854   +57     
   - Misses        39612    39648   +36     
   - Partials       5723     5724    +1     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.41% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.51% <0.00%> (+<0.01%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
   
   ... and [16 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3959417375

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
       "triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
       "triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "triggerType" : "PUSH"
     }, {
       "hash" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12293",
       "triggerID" : "cbe77a8b82bd7b752a9364f75fa32e409b7539eb",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * cbe77a8b82bd7b752a9364f75fa32e409b7539eb Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12293) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2850652704


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java:
##########
@@ -1325,6 +1325,15 @@ public class FlinkOptions extends HoodieConfig {
               + "Each task manager will create a unique subdirectory under this path. "
               + "The directory is cleaned up when the lookup function is closed.");
 
+  public static final ConfigOption<String> LOOKUP_PARTITIONS =
+      key("lookup.partitions")

Review Comment:
   it looks like dim table already supports filer push down through SQL: https://cwiki.apache.org/confluence/display/FLINK/FLIP-324%3A+Introduce+Runtime+Filter+for+Flink+Batch+Jobs, we can utilize the SQL filters instead of the explicit options.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944753261

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 02979148bf26522bbb63b4128e06a516684e37cf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148) 
   * 1f1bb42055b7f176cb4d4d2bd614de433d99740a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3948912987

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219) 
   * 0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949582942

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
       "triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
       "triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251) 
   * f1ea3e2b59bfa373ccca2abf12004acb221e8639 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945031725

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 55.44%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   55.44%   -1.85%     
   + Complexity    18532    17882     -650     
   ============================================
     Files          1944     1944              
     Lines        106132   106157      +25     
     Branches      13118    13124       +6     
   ============================================
   - Hits          60797    58858    -1939     
   - Misses        39612    41496    +1884     
   - Partials       5723     5803      +80     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.37% <0.00%> (-4.06%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
   
   ... and [254 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2850644442


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java:
##########
@@ -403,16 +405,45 @@ public LookupRuntimeProvider getLookupRuntimeProvider(LookupContext context) {
     Duration duration = conf.get(LOOKUP_JOIN_CACHE_TTL);
     boolean asyncEnabled = conf.get(LOOKUP_ASYNC);
     int asyncThreadNumber = conf.get(LOOKUP_ASYNC_THREAD_NUMBER);
+    Option<PartitionPruners.PartitionPruner> lookupPruner = buildLookupPartitionPruner();
     return LookupRuntimeProviderFactory.create(
         new HoodieLookupFunction(
-            new HoodieLookupTableReader(this::getBatchInputFormat, conf),
+            new HoodieLookupTableReader(
+                lookupPruner.isPresent()
+                    ? () -> getBatchInputFormatWithPruner(lookupPruner.get())
+                    : this::getBatchInputFormat,
+                conf),
             (RowType) getProducedDataType().notNull().getLogicalType(),
             getLookupKeys(context.getKeys()),
             duration,
             conf
         ), asyncEnabled, asyncThreadNumber);
   }
 
+  private Option<PartitionPruners.PartitionPruner> buildLookupPartitionPruner() {
+    if (!conf.contains(LOOKUP_PARTITIONS) || !isPartitioned()) {
+      return Option.empty();
+    }
+    List<String> partitionPaths = PartitionPathParser.parseLookupPartitionPaths(
+        conf.get(LOOKUP_PARTITIONS), this.partitionKeys, conf.get(FlinkOptions.HIVE_STYLE_PARTITIONING));
+    if (partitionPaths.isEmpty()) {
+      return Option.empty();
+    }
+    return Option.of(PartitionPruners.builder()
+        .candidatePartitions(partitionPaths)
+        .build());
+  }
+
+  private InputFormat<RowData, ?> getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner lookupPruner) {
+    PartitionPruners.PartitionPruner saved = this.partitionPruner;
+    this.partitionPruner = lookupPruner;
+    try {
+      return getBatchInputFormat();
+    } finally {
+      this.partitionPruner = saved;

Review Comment:
   you are right, the original code looks simpler, we can revert to the old impl as long as we can ensure the file index is not reused for other scan sources. maybe we just nullify the file index manually here to make it safe.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2840897721


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java:
##########
@@ -403,16 +405,45 @@ public LookupRuntimeProvider getLookupRuntimeProvider(LookupContext context) {
     Duration duration = conf.get(LOOKUP_JOIN_CACHE_TTL);
     boolean asyncEnabled = conf.get(LOOKUP_ASYNC);
     int asyncThreadNumber = conf.get(LOOKUP_ASYNC_THREAD_NUMBER);
+    Option<PartitionPruners.PartitionPruner> lookupPruner = buildLookupPartitionPruner();
     return LookupRuntimeProviderFactory.create(
         new HoodieLookupFunction(
-            new HoodieLookupTableReader(this::getBatchInputFormat, conf),
+            new HoodieLookupTableReader(
+                lookupPruner.isPresent()
+                    ? () -> getBatchInputFormatWithPruner(lookupPruner.get())
+                    : this::getBatchInputFormat,
+                conf),
             (RowType) getProducedDataType().notNull().getLogicalType(),
             getLookupKeys(context.getKeys()),
             duration,
             conf
         ), asyncEnabled, asyncThreadNumber);
   }
 
+  private Option<PartitionPruners.PartitionPruner> buildLookupPartitionPruner() {
+    if (!conf.contains(LOOKUP_PARTITIONS) || !isPartitioned()) {
+      return Option.empty();
+    }
+    List<String> partitionPaths = PartitionPathParser.parseLookupPartitionPaths(
+        conf.get(LOOKUP_PARTITIONS), this.partitionKeys, conf.get(FlinkOptions.HIVE_STYLE_PARTITIONING));
+    if (partitionPaths.isEmpty()) {
+      return Option.empty();
+    }
+    return Option.of(PartitionPruners.builder()
+        .candidatePartitions(partitionPaths)
+        .build());
+  }
+
+  private InputFormat<RowData, ?> getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner lookupPruner) {
+    PartitionPruners.PartitionPruner saved = this.partitionPruner;
+    this.partitionPruner = lookupPruner;
+    try {
+      return getBatchInputFormat();
+    } finally {
+      this.partitionPruner = saved;

Review Comment:
   @danny0405 Ok, i've refactored this code (in separate commit). But it was not enough to just pass around PartitionPruner into getBatchInputFormat, because it is used deeper. (in `FileIndexReader`)  I think, now it's not less error-prone, but more complicated.  
   Review it please, if it's ok for you, approve it, then I'll actualize PR description and merge.  
   If during review you decide that the first version was better - let me know, i'll drop the second commit and re-request review.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945026880

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 51.81%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
   > 
   > <details><summary>HEAD has 7 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (5b7100c) |
   >|------|------|------|
   >|spark-scala-tests|6|5|
   >|spark-java-tests|9|3|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   51.81%   -5.47%     
   + Complexity    18532    16561    -1971     
   ============================================
     Files          1944     1944              
     Lines        106132   106157      +25     
     Branches      13118    13124       +6     
   ============================================
   - Hits          60797    55009    -5788     
   - Misses        39612    45248    +5636     
   - Partials       5723     5900     +177     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.23% <0.00%> (-14.20%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.42% <0.00%> (-0.09%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
   
   ... and [506 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945039291

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 55.47%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   55.47%   -1.82%     
   + Complexity    18532    17890     -642     
   ============================================
     Files          1944     1944              
     Lines        106132   106157      +25     
     Branches      13118    13124       +6     
   ============================================
   - Hits          60797    58887    -1910     
   - Misses        39612    41469    +1857     
   - Partials       5723     5801      +78     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.42% <0.00%> (-4.01%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
   
   ... and [251 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945049304

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 02979148bf26522bbb63b4128e06a516684e37cf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148) 
   * 1f1bb42055b7f176cb4d4d2bd614de433d99740a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215) 
   * 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949161545

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 51.71%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
   > 
   > <details><summary>HEAD has 8 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
   >|------|------|------|
   >|spark-scala-tests|6|4|
   >|spark-java-tests|9|3|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   51.71%   -5.58%     
   + Complexity    18532    16532    -2000     
   ============================================
     Files          1944     1944              
     Lines        106132   106164      +32     
     Branches      13118    13126       +8     
   ============================================
   - Hits          60797    54898    -5899     
   - Misses        39612    45353    +5741     
   - Partials       5723     5913     +190     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.22% <0.00%> (-14.20%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.27% <0.00%> (-0.24%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
   
   ... and [527 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949171599

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 55.30%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   55.30%   -1.98%     
   + Complexity    18532    17856     -676     
   ============================================
     Files          1944     1944              
     Lines        106132   106164      +32     
     Branches      13118    13126       +8     
   ============================================
   - Hits          60797    58718    -2079     
   - Misses        39612    41662    +2050     
   - Partials       5723     5784      +61     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.20% <0.00%> (-4.23%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
   
   ... and [269 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944835871

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 45.39%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
   > 
   > <details><summary>HEAD has 15 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (5b7100c) |
   >|------|------|------|
   >|spark-scala-tests|6|0|
   >|spark-java-tests|9|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #18229       +/-   ##
   =============================================
   - Coverage     57.28%   45.39%   -11.89%     
   + Complexity    18532     8370    -10162     
   =============================================
     Files          1944     1177      -767     
     Lines        106132    60264    -45868     
     Branches      13118     6420     -6698     
   =============================================
   - Hits          60797    27357    -33440     
   + Misses        39612    29933     -9679     
   + Partials       5723     2974     -2749     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `17.20% <0.00%> (-9.27%)` | :arrow_down: |
   
   ... and [1381 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2840897721


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java:
##########
@@ -403,16 +405,45 @@ public LookupRuntimeProvider getLookupRuntimeProvider(LookupContext context) {
     Duration duration = conf.get(LOOKUP_JOIN_CACHE_TTL);
     boolean asyncEnabled = conf.get(LOOKUP_ASYNC);
     int asyncThreadNumber = conf.get(LOOKUP_ASYNC_THREAD_NUMBER);
+    Option<PartitionPruners.PartitionPruner> lookupPruner = buildLookupPartitionPruner();
     return LookupRuntimeProviderFactory.create(
         new HoodieLookupFunction(
-            new HoodieLookupTableReader(this::getBatchInputFormat, conf),
+            new HoodieLookupTableReader(
+                lookupPruner.isPresent()
+                    ? () -> getBatchInputFormatWithPruner(lookupPruner.get())
+                    : this::getBatchInputFormat,
+                conf),
             (RowType) getProducedDataType().notNull().getLogicalType(),
             getLookupKeys(context.getKeys()),
             duration,
             conf
         ), asyncEnabled, asyncThreadNumber);
   }
 
+  private Option<PartitionPruners.PartitionPruner> buildLookupPartitionPruner() {
+    if (!conf.contains(LOOKUP_PARTITIONS) || !isPartitioned()) {
+      return Option.empty();
+    }
+    List<String> partitionPaths = PartitionPathParser.parseLookupPartitionPaths(
+        conf.get(LOOKUP_PARTITIONS), this.partitionKeys, conf.get(FlinkOptions.HIVE_STYLE_PARTITIONING));
+    if (partitionPaths.isEmpty()) {
+      return Option.empty();
+    }
+    return Option.of(PartitionPruners.builder()
+        .candidatePartitions(partitionPaths)
+        .build());
+  }
+
+  private InputFormat<RowData, ?> getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner lookupPruner) {
+    PartitionPruners.PartitionPruner saved = this.partitionPruner;
+    this.partitionPruner = lookupPruner;
+    try {
+      return getBatchInputFormat();
+    } finally {
+      this.partitionPruner = saved;

Review Comment:
   @danny0405 Ok, i've refactored this code (in separate commit). Agree with you about error-prone, but it was not enough to just pass around PartitionPruner into getBatchInputFormat, because it is used deeper (in `FileIndexReader`).  I think, now it becomes not less error-prone, but more complicated.  
   Review it please, if it's ok for you, approve it, then I'll actualize PR description and merge.  
   If during review you decide that the first version was better - let me know, i'll drop the second commit and re-request review.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944767076

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 02979148bf26522bbb63b4128e06a516684e37cf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148) 
   * 1f1bb42055b7f176cb4d4d2bd614de433d99740a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215) 
   * 5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3950145810

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     }, {
       "hash" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12215",
       "triggerID" : "1f1bb42055b7f176cb4d4d2bd614de433d99740a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12219",
       "triggerID" : "5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12251",
       "triggerID" : "0dc55ee0e529f881d8170dbb670fd1ac5c3f1f6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258",
       "triggerID" : "f1ea3e2b59bfa373ccca2abf12004acb221e8639",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f1ea3e2b59bfa373ccca2abf12004acb221e8639 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12258) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949165258

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 51.86%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
   > 
   > <details><summary>HEAD has 6 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
   >|------|------|------|
   >|spark-java-tests|9|3|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   51.86%   -5.43%     
   + Complexity    18532    16577    -1955     
   ============================================
     Files          1944     1944              
     Lines        106132   106164      +32     
     Branches      13118    13126       +8     
   ============================================
   - Hits          60797    55057    -5740     
   - Misses        39612    45201    +5589     
   - Partials       5723     5906     +183     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.22% <0.00%> (-14.20%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
   
   ... and [501 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949138791

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 43.53%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
   > 
   > <details><summary>HEAD has 9 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
   >|------|------|------|
   >|spark-scala-tests|6|3|
   >|spark-java-tests|9|3|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #18229       +/-   ##
   =============================================
   - Coverage     57.28%   43.53%   -13.76%     
   + Complexity    18532    13836     -4696     
   =============================================
     Files          1944     1944               
     Lines        106132   106164       +32     
     Branches      13118    13126        +8     
   =============================================
   - Hits          60797    46216    -14581     
   - Misses        39612    54651    +15039     
   + Partials       5723     5297      -426     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.22% <0.00%> (-14.20%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
   
   ... and [784 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932500079

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8d6d98805ddabfa1da4ff4df6304984b37ec6982 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147) 
   * 02979148bf26522bbb63b4128e06a516684e37cf Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12148) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3932208141

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147",
       "triggerID" : "8d6d98805ddabfa1da4ff4df6304984b37ec6982",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "02979148bf26522bbb63b4128e06a516684e37cf",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8d6d98805ddabfa1da4ff4df6304984b37ec6982 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=12147) 
   * 02979148bf26522bbb63b4128e06a516684e37cf UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2838975681


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java:
##########
@@ -403,16 +405,45 @@ public LookupRuntimeProvider getLookupRuntimeProvider(LookupContext context) {
     Duration duration = conf.get(LOOKUP_JOIN_CACHE_TTL);
     boolean asyncEnabled = conf.get(LOOKUP_ASYNC);
     int asyncThreadNumber = conf.get(LOOKUP_ASYNC_THREAD_NUMBER);
+    Option<PartitionPruners.PartitionPruner> lookupPruner = buildLookupPartitionPruner();
     return LookupRuntimeProviderFactory.create(
         new HoodieLookupFunction(
-            new HoodieLookupTableReader(this::getBatchInputFormat, conf),
+            new HoodieLookupTableReader(
+                lookupPruner.isPresent()
+                    ? () -> getBatchInputFormatWithPruner(lookupPruner.get())
+                    : this::getBatchInputFormat,
+                conf),
             (RowType) getProducedDataType().notNull().getLogicalType(),
             getLookupKeys(context.getKeys()),
             duration,
             conf
         ), asyncEnabled, asyncThreadNumber);
   }
 
+  private Option<PartitionPruners.PartitionPruner> buildLookupPartitionPruner() {
+    if (!conf.contains(LOOKUP_PARTITIONS) || !isPartitioned()) {
+      return Option.empty();
+    }
+    List<String> partitionPaths = PartitionPathParser.parseLookupPartitionPaths(
+        conf.get(LOOKUP_PARTITIONS), this.partitionKeys, conf.get(FlinkOptions.HIVE_STYLE_PARTITIONING));
+    if (partitionPaths.isEmpty()) {
+      return Option.empty();
+    }
+    return Option.of(PartitionPruners.builder()
+        .candidatePartitions(partitionPaths)
+        .build());
+  }
+
+  private InputFormat<RowData, ?> getBatchInputFormatWithPruner(PartitionPruners.PartitionPruner lookupPruner) {
+    PartitionPruners.PartitionPruner saved = this.partitionPruner;
+    this.partitionPruner = lookupPruner;
+    try {
+      return getBatchInputFormat();
+    } finally {
+      this.partitionPruner = saved;

Review Comment:
   this is error prone, just pass around the `PartitionPruner` as a param into `getBatchInputFormat` ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3945028225

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 51.86%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
   > 
   > <details><summary>HEAD has 6 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (5b7100c) |
   >|------|------|------|
   >|spark-java-tests|9|3|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   51.86%   -5.43%     
   + Complexity    18532    16578    -1954     
   ============================================
     Files          1944     1944              
     Lines        106132   106157      +25     
     Branches      13118    13124       +6     
   ============================================
   - Hits          60797    55057    -5740     
   - Misses        39612    45198    +5586     
   - Partials       5723     5902     +179     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.23% <0.00%> (-14.20%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `19.35% <0.00%> (-7.12%)` | :arrow_down: |
   
   ... and [498 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949175801

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 55.50%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18229      +/-   ##
   ============================================
   - Coverage     57.28%   55.50%   -1.78%     
   + Complexity    18532    17911     -621     
   ============================================
     Files          1944     1944              
     Lines        106132   106164      +32     
     Branches      13118    13126       +8     
   ============================================
   - Hits          60797    58930    -1867     
   - Misses        39612    41439    +1827     
   - Partials       5723     5795      +72     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.47% <0.00%> (-3.95%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.49% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `18.94% <0.00%> (-7.53%)` | :arrow_down: |
   
   ... and [243 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944971116

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 43.53%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
   > 
   > <details><summary>HEAD has 9 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (5b7100c) |
   >|------|------|------|
   >|spark-scala-tests|6|3|
   >|spark-java-tests|9|3|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #18229       +/-   ##
   =============================================
   - Coverage     57.28%   43.53%   -13.75%     
   + Complexity    18532    13837     -4695     
   =============================================
     Files          1944     1944               
     Lines        106132   106157       +25     
     Branches      13118    13124        +6     
   =============================================
   - Hits          60797    46217    -14580     
   - Misses        39612    54646    +15034     
   + Partials       5723     5294      -429     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.23% <0.00%> (-14.20%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `17.20% <0.00%> (-9.27%)` | :arrow_down: |
   
   ... and [782 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3944957018

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 39.70%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b7100c`](https://app.codecov.io/gh/apache/hudi/commit/5b7100cd3bbac92f48b1f1f96d4bd534449aaf3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [25 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (5b7100c). Click for more details.
   > 
   > <details><summary>HEAD has 12 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (5b7100c) |
   >|------|------|------|
   >|spark-scala-tests|6|3|
   >|spark-java-tests|9|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #18229       +/-   ##
   =============================================
   - Coverage     57.28%   39.70%   -17.58%     
   + Complexity    18532    12574     -5958     
   =============================================
     Files          1944     1944               
     Lines        106132   106157       +25     
     Branches      13118    13124        +6     
   =============================================
   - Hits          60797    42153    -18644     
   - Misses        39612    58945    +19333     
   + Partials       5723     5059      -664     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.39% <0.00%> (-0.02%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `17.20% <0.00%> (-9.27%)` | :arrow_down: |
   
   ... and [856 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949124126

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 43.49%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
   > 
   > <details><summary>HEAD has 10 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
   >|------|------|------|
   >|spark-scala-tests|6|3|
   >|spark-java-tests|9|2|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #18229       +/-   ##
   =============================================
   - Coverage     57.28%   43.49%   -13.80%     
   + Complexity    18532    13823     -4709     
   =============================================
     Files          1944     1944               
     Lines        106132   106164       +32     
     Branches      13118    13126        +8     
   =============================================
   - Hits          60797    46171    -14626     
   - Misses        39612    54691    +15079     
   + Partials       5723     5302      -421     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.67% <0.00%> (-13.76%)` | :arrow_down: |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
   
   ... and [786 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3949098386

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 39.70%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
   > 
   > <details><summary>HEAD has 12 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
   >|------|------|------|
   >|spark-scala-tests|6|3|
   >|spark-java-tests|9|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #18229       +/-   ##
   =============================================
   - Coverage     57.28%   39.70%   -17.58%     
   + Complexity    18532    12575     -5957     
   =============================================
     Files          1944     1944               
     Lines        106132   106164       +32     
     Branches      13118    13126        +8     
   =============================================
   - Hits          60797    42155    -18642     
   - Misses        39612    58949    +19337     
   + Partials       5723     5060      -663     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.15% <0.00%> (-12.36%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
   
   ... and [858 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2852815343


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java:
##########
@@ -1325,6 +1325,15 @@ public class FlinkOptions extends HoodieConfig {
               + "Each task manager will create a unique subdirectory under this path. "
               + "The directory is cleaned up when the lookup function is closed.");
 
+  public static final ConfigOption<String> LOOKUP_PARTITIONS =
+      key("lookup.partitions")

Review Comment:
   Sorry, i don't understand your point. Could you please explain your idea?
   Why using SQL filters are better than pruning partitions directly?  
   Or you mean, that users prefer to write SQL with "... where dim.partitionField='par1' or dim.partitionField='par2'"? then this whole PR is not needed at all.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #18229:
URL: https://github.com/apache/hudi/pull/18229#issuecomment-3948973952

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `27 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 45.40%. Comparing base ([`8c5f832`](https://app.codecov.io/gh/apache/hudi/commit/8c5f83237c11f6b67f99e56fb06881312e39a625?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f1ea3e2`](https://app.codecov.io/gh/apache/hudi/commit/f1ea3e2b59bfa373ccca2abf12004acb221e8639?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | 0.00% | [27 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (8c5f832) and HEAD (f1ea3e2). Click for more details.
   > 
   > <details><summary>HEAD has 15 uploads less than BASE</summary>
   >
   >| Flag | BASE (8c5f832) | HEAD (f1ea3e2) |
   >|------|------|------|
   >|spark-scala-tests|6|0|
   >|spark-java-tests|9|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #18229       +/-   ##
   =============================================
   - Coverage     57.28%   45.40%   -11.89%     
   + Complexity    18532     8372    -10160     
   =============================================
     Files          1944     1177      -767     
     Lines        106132    60266    -45866     
     Branches      13118     6421     -6697     
   =============================================
   - Hits          60797    27362    -33435     
   + Misses        39612    29930     -9682     
   + Partials       5723     2974     -2749     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `45.40% <0.00%> (-0.01%)` | :arrow_down: |
   | [spark-java-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [spark-scala-tests](https://app.codecov.io/gh/apache/hudi/pull/18229/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/18229?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../apache/hudi/common/table/PartitionPathParser.java](https://app.codecov.io/gh/apache/hudi/pull/18229?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcommon%2Ftable%2FPartitionPathParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3RhYmxlL1BhcnRpdGlvblBhdGhQYXJzZXIuamF2YQ==) | `16.84% <0.00%> (-9.63%)` | :arrow_down: |
   
   ... and [1383 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/18229/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(flink): lookup join caches only specified partitions of dimension table [hudi]

Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #18229:
URL: https://github.com/apache/hudi/pull/18229#discussion_r2844245487


##########
hudi-common/src/main/java/org/apache/hudi/common/table/PartitionPathParser.java:
##########
@@ -164,4 +171,53 @@ private static Object inferDateValue(
   private static boolean isTimeBasedType(HoodieSchemaType type) {
     return type == HoodieSchemaType.DATE || type == HoodieSchemaType.TIMESTAMP || type == HoodieSchemaType.TIME;
   }
+
+  /**
+   * Parses the {@code lookup.partitions} option value into a list of Hudi partition paths.
+   *
+   * <p>The spec format is {@code key1=val1,key2=val2} per partition, with multiple partitions
+   * separated by {@code ;}. Example: {@code "dt=2024-01-01,region=us;dt=2024-01-02,region=eu"}.
+   *
+   * @param spec         the raw option value
+   * @param partitionKeys ordered list of partition key names as defined in the table schema
+   * @param hiveStyle    whether the table uses Hive-style partitioning ({@code key=value} directories)
+   * @return list of partition paths in the format used by Hudi's file index
+   * @throws IllegalArgumentException if any key in the spec is not a valid partition key,
+   *                                  or if a key-value pair does not follow {@code key=value} format
+   */
+  public static List<String> parseLookupPartitionPaths(String spec, List<String> partitionKeys, boolean hiveStyle) {

Review Comment:
   ok, i'll do it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org