[I] [Bug] For primary key table, when use spark insert overwrite some partition and the compact behavior is triggered, IllegalStateException error occurred [incubator-paimon]

["Xiangakun (via GitHub)" <gi...@apache.org>]

Xiangakun opened a new issue, #2425:
URL: https://github.com/apache/incubator-paimon/issues/2425

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
   
   
   ### Paimon version
   
   release-0.5
   
   ### Compute Engine
   
   Spark
   
   ### Minimal reproduce step
   
   1. create table paimon.paimon_test.test_table(id bigint, data string, dt string) partitioned by (dt) TBLPROPERTIES('num-sorted-run.compaction-trigger' = '1', 'primary-key' = 'dt,id');
   2. insert into  paimon.paimon_test.test_table values (1, 'test', '2023-11-30');
   3. insert overwrite paimon.paimon_test.test_table partition (dt = '2023-11-30') select id,data from paimon.paimon_test.test_table  where dt = '2023-11-30';
   
   ### What doesn't meet your expectations?
   
   Exception throw:
   Caused by: java.lang.IllegalStateException: Trying to delete file data-44539a97-5060-4b5f-9ea1-9f9119452f50-0.parquet which is not previously added. Manifest might be corrupted.
   	at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204)
   	at org.apache.paimon.manifest.ManifestEntry.assertNoDelete(ManifestEntry.java:154)
   	at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:810)
   
   **the conflict file is the one which added by step two**
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] For primary key table, when use spark insert overwrite some partition and the compact behavior is triggered, IllegalStateException error occurred [incubator-paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi closed issue #2425: [Bug] For primary key table, when use spark insert overwrite some partition and the compact behavior is triggered, IllegalStateException error occurred
URL: https://github.com/apache/incubator-paimon/issues/2425


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org