You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "JingsongLi (via GitHub)" <gi...@apache.org> on 2026/06/01 14:40:58 UTC
[PR] [core] Optimize deletion vector index scan [paimon]
JingsongLi opened a new pull request, #8066:
URL: https://github.com/apache/paimon/pull/8066
## Summary
Optimize deletion vector index scanning during snapshot planning by avoiding unnecessary manifest reads and limiting DV index scans to the requested partition-bucket pairs.
## Changes
- Return early from when there are no requested buckets or all requested buckets are served from the DV meta cache.
- Add exact partition-bucket scan support in , using row-level filtering for index type, partition, and bucket before deserializing matching index manifest entries.
- Add index manifest row getters for efficient row-level filtering.
- Cover exact bucket scanning with a focused unit test, including empty input and same-bucket different-index-type cases.
## Testing
- [INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-common_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-common_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-common/pom.xml, line 33, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-ut_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-ut_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-ut/pom.xml, line 31, column 17
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: aarch_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 26.3
[INFO] os.detected.version.major: 26
[INFO] os.detected.version.minor: 3
[INFO] os.detected.classifier: osx-aarch_64
[INFO]
[INFO] -------------------< org.apache.paimon:paimon-core >--------------------
[INFO] Building Paimon : Core 1.5-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- checkstyle:2.17:check (validate) @ paimon-core ---
[INFO]
[INFO] --- spotless:2.13.0:check (spotless-check) @ paimon-core ---
[INFO] Spotless check skipped
[INFO] Spotless check skipped
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven-version) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-jackson) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (forbid-log4j-1) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (forbid-direct-table-planner-dependencies) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- directory:1.0:directory-of (directories) @ paimon-core ---
[INFO] Directory of org.apache.paimon:paimon-parent set to: /Users/lijingsong/IdeaProjects/paimon
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @ paimon-core ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- resources:3.1.0:resources (default-resources) @ paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:compile (default-compile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- resources:3.1.0:testResources (default-testResources) @ paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 29 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:testCompile (default-testCompile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- surefire:3.0.0-M5:test (default-test) @ paimon-core ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.paimon.index.IndexFileHandlerTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.876 s - in org.apache.paimon.index.IndexFileHandlerTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.833 s
[INFO] Finished at: 2026-06-01T22:40:30+08:00
[INFO] ------------------------------------------------------------------------
- [INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-common_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-common_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-common/pom.xml, line 33, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-ut_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-ut_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-ut/pom.xml, line 31, column 17
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: aarch_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 26.3
[INFO] os.detected.version.major: 26
[INFO] os.detected.version.minor: 3
[INFO] os.detected.classifier: osx-aarch_64
[INFO]
[INFO] -------------------< org.apache.paimon:paimon-core >--------------------
[INFO] Building Paimon : Core 1.5-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- checkstyle:2.17:check (validate) @ paimon-core ---
[INFO]
[INFO] --- spotless:2.13.0:check (spotless-check) @ paimon-core ---
[INFO] Spotless check skipped
[INFO] Spotless check skipped
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven-version) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-jackson) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (forbid-log4j-1) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (forbid-direct-table-planner-dependencies) @ paimon-core ---
[INFO] Skipping Rule Enforcement.
[INFO]
[INFO] --- directory:1.0:directory-of (directories) @ paimon-core ---
[INFO] Directory of org.apache.paimon:paimon-parent set to: /Users/lijingsong/IdeaProjects/paimon
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @ paimon-core ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- resources:3.1.0:resources (default-resources) @ paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:compile (default-compile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- resources:3.1.0:testResources (default-testResources) @ paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 29 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:testCompile (default-testCompile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- surefire:3.0.0-M5:test (default-test) @ paimon-core ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.paimon.table.PrimaryKeySimpleTableTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.95 s - in org.apache.paimon.table.PrimaryKeySimpleTableTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.424 s
[INFO] Finished at: 2026-06-01T22:40:36+08:00
[INFO] ------------------------------------------------------------------------
- [INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-common_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-common_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-common/pom.xml, line 33, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-ut_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-ut_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-ut/pom.xml, line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark3-common_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark3-common_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark3-common/pom.xml, line 33, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-3.5_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-3.5_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.5/pom.xml, line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-3.4_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-3.4_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.4/pom.xml, line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-3.3_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-3.3_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.3/pom.xml, line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-3.2_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-3.2_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.2/pom.xml, line 31, column 17
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: aarch_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 26.3
[INFO] os.detected.version.major: 26
[INFO] os.detected.version.minor: 3
[INFO] os.detected.classifier: osx-aarch_64
[INFO]
[INFO] -------------------< org.apache.paimon:paimon-core >--------------------
[INFO] Building Paimon : Core 1.5-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- checkstyle:2.17:check (validate) @ paimon-core ---
[INFO]
[INFO] --- spotless:2.13.0:check (spotless-check) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven-version) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-jackson) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (forbid-log4j-1) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (forbid-direct-table-planner-dependencies) @ paimon-core ---
[INFO]
[INFO] --- directory:1.0:directory-of (directories) @ paimon-core ---
[INFO] Directory of org.apache.paimon:paimon-parent set to: /Users/lijingsong/IdeaProjects/paimon
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @ paimon-core ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- resources:3.1.0:resources (default-resources) @ paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:compile (default-compile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.049 s
[INFO] Finished at: 2026-06-01T22:40:52+08:00
[INFO] ------------------------------------------------------------------------
-
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Optimize deletion vector index scan [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8066:
URL: https://github.com/apache/paimon/pull/8066#issuecomment-4598297982
Addressed the review comments in the latest commit:
- now copies the input bucket set into a local before consuming cache hits, so future callers will not observe mutations.
- is already thread-safe through double-checked synchronization, and I added a concurrent regression test to verify that a lazy supplier is initialized only once under concurrent reads.
Verified with:
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-common_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-common_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-common/pom.xml, line 33, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-ut_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-ut_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-ut/pom.xml, line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark3-common_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark3-common_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark3-common/pom.xml, line 33, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-3.5_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-3.5_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.5/pom.xml, line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-3.4_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-3.4_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.4/pom.xml, line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-3.3_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-3.3_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.3/pom.xml, line 31, column 17
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.paimon:paimon-spark-3.2_2.12:jar:1.5-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.paimon:paimon-spark-3.2_${scala.binary.version}:1.5-SNAPSHOT, /Users/lijingsong/IdeaProjects/paimon/paimon-spark/paimon-spark-3.2/pom.xml, line 31, column 17
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: aarch_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 26.3
[INFO] os.detected.version.major: 26
[INFO] os.detected.version.minor: 3
[INFO] os.detected.classifier: osx-aarch_64
[INFO]
[INFO] -------------------< org.apache.paimon:paimon-core >--------------------
[INFO] Building Paimon : Core 1.5-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- checkstyle:2.17:check (validate) @ paimon-core ---
[INFO]
[INFO] --- spotless:2.13.0:check (spotless-check) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven-version) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (enforce-maven) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (ban-unsafe-jackson) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (forbid-log4j-1) @ paimon-core ---
[INFO]
[INFO] --- enforcer:3.0.0-M1:enforce (forbid-direct-table-planner-dependencies) @ paimon-core ---
[INFO]
[INFO] --- directory:1.0:directory-of (directories) @ paimon-core ---
[INFO] Directory of org.apache.paimon:paimon-parent set to: /Users/lijingsong/IdeaProjects/paimon
[INFO]
[INFO] --- remote-resources:1.7.0:process (process-resource-bundles) @ paimon-core ---
[INFO] Preparing remote bundle org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- resources:3.1.0:resources (default-resources) @ paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:compile (default-compile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- resources:3.1.0:testResources (default-testResources) @ paimon-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 29 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- compiler:3.8.0:testCompile (default-testCompile) @ paimon-core ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- surefire:3.0.0-M5:test (default-test) @ paimon-core ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.paimon.utils.DVMetaCacheTest
[INFO] Running org.apache.paimon.index.IndexFileHandlerTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.081 s - in org.apache.paimon.utils.DVMetaCacheTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.046 s - in org.apache.paimon.index.IndexFileHandlerTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.011 s
[INFO] Finished at: 2026-06-02T10:53:01+08:00
[INFO] ------------------------------------------------------------------------
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Optimize deletion vector index scan [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8066:
URL: https://github.com/apache/paimon/pull/8066#issuecomment-4598344036
Thanks, addressed this in the latest update. `LazyDeletionFilesField#get()` is now synchronized, so the DV meta cache wrapper itself explicitly serializes lazy initialization.
Verified again with:
`mvn -pl paimon-core -Dtest=DVMetaCacheTest,IndexFileHandlerTest test`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Optimize deletion vector index scan [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #8066:
URL: https://github.com/apache/paimon/pull/8066
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Optimize deletion vector index scan [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8066:
URL: https://github.com/apache/paimon/pull/8066#issuecomment-4594238290
But now the caching efficiency of dvMetaCache is very low. If a partition has thousands of buckets and only caches a little at a time, it needs to be read repeatedly. Is it possible to install a primitive Lazied class (such as LazyField) in DVMetaCache, which can cache bytes while delaying execution?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org