You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "JingsongLi (via GitHub)" <gi...@apache.org> on 2026/07/01 08:22:34 UTC
[PR] [spark] Fix data evolution update with global index [paimon]
JingsongLi opened a new pull request, #8411:
URL: https://github.com/apache/paimon/pull/8411
## Summary
Fixes Data Evolution V1 UPDATE scans with global index by forcing matched-update row-id discovery to use full global-index search on the current snapshot. This prevents rows inserted after global index creation from being skipped during UPDATE.
## Changes
- Adds a shared matched-update scan option helper for Data Evolution MERGE commands.
- Applies `global-index.search-mode=full` and `scan.snapshot-id=<latest>` before UPDATE builds its row-id source.
- Adds a regression test covering V1 UPDATE against rows inserted after BTree global index creation.
## Testing
- [x] `mvn -pl paimon-spark/paimon-spark-3.5 -am -Pspark3,fast-build -DfailIfNoTests=false -DwildcardSuites=org.apache.paimon.spark.sql.RowTrackingTest -Dtest=none test`
- [x] `JAVA_HOME=$(/usr/libexec/java_home -v 17) mvn -pl paimon-spark/paimon-spark-4.0 -am -Pspark4,fast-build -DskipTests compile`
- [x] `git diff --check`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [spark] Fix data evolution matched updates with global index [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #8411:
URL: https://github.com/apache/paimon/pull/8411
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [spark] Fix data evolution matched updates with global index [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8411:
URL: https://github.com/apache/paimon/pull/8411#issuecomment-4852255538
Addressed the Spotless review comment in the latest head `41d37c42a`.
Local verification:
- `mvn -pl paimon-spark/paimon-spark-common -am -DskipTests -DfailIfNoTests=false spotless:check`
- `JAVA_HOME=$(/usr/libexec/java_home -v 17) mvn -pl paimon-spark/paimon-spark-4.0 -am -Pspark4 -DskipTests -DfailIfNoTests=false spotless:check`
- `git diff --check`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org