You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "ASiegeLion (via GitHub)" <gi...@apache.org> on 2025/08/25 03:14:42 UTC
[I] [Bug] delete file error [paimon]
ASiegeLion opened a new issue, #6137:
URL: https://github.com/apache/paimon/issues/6137
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
paimon 0.8
### Compute Engine
flink 1.17.2
### Minimal reproduce step
paimon table ：
```
CREATE TABLE paimon.hainadw.yydw_dm_entity_person_paimon_mini(
`hdid` string,
`attribute` string,
`last_update_dt` string,
`proctime` AS PROCTIME(),
PRIMARY KEY(`hdid`) NOT ENFORCED
) COMMENT ''
WITH (
'num-sorted-run.stop-trigger' = '2147483647',
'delete.force-produce-changelog' = 'false',
'changelog-producer' = 'input',
'sort-spill-threshold' = '10',
'bucket' = '64',
'log.scan.remove-normalize' = 'true',
'snapshot.time-retained' = '24h',
'log.consistency' = 'eventual',
'metastore.partitioned-table' = 'true'
)
```
There are two flink tasks writing to this table：
1 flink stream sql:
```
insert into paimon.hainadw.yydw_dm_entity_person_paimon_mini
select
hdid,
CleanJsonString(attribute),
date_format(current_date, 'yyyy-MM-dd') as last_update_dt
from new_device_after_fetch
where status = 'SUCCESS'
and attribute > '';
```
2 flink batch sql:
```
delete from paimon.hainadw.yydw_dm_entity_person_paimon_mini/*+ OPTIONS('write-only'='true') */
where last_update_dt < TIMESTAMPADD(DAY, -366, CURRENT_DATE);
```
### What doesn't meet your expectations?
2025-08-23 08:07:07,925 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_0) switched from RUNNING to FAILED on container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473).
java.lang.RuntimeException: File deletion conflicts detected! Give up committing.
Don't panic!
Conflicts during commits are normal and this failure is intended to resolve the conflicts.
Conflicts are mainly caused by the following scenarios:
1. Your job is suffering from back-pressuring.
There are too many snapshots waiting to be committed and an exception occurred during the commit procedure (most probably due to checkpoint timeout).
See https://paimon.apache.org/docs/master/maintenance/write-performance/ for how to improve writing performance.
2. Multiple jobs are writing into the same partition at the same time, or you use STATEMENT SET to execute multiple INSERT statements into the same Paimon table.
You'll probably see different base commit user and current commit user below.
You can use https://paimon.apache.org/docs/master/maintenance/dedicated-compaction#dedicated-compaction-jobto support multiple writing.
3. You're recovering from an old savepoint, or you're creating multiple jobs from a savepoint.
The job will fail continuously in this scenario to protect metadata from corruption.
You can either recover from the latest savepoint, or you can revert the table to the snapshot corresponding to the old savepoint.
Base commit user is: 5a90b504-8a73-43dc-871a-2e82d388ab37; Current commit user is: 5a90b504-8a73-43dc-871a-2e82d388ab37
Base entries are:
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-0.orc', minKey=org.apache.paimon.data.BinaryRow@8689bba1, maxKey=org.apache.paimon.data.BinaryRow@c328c0a1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-1.orc', minKey=org.apache.paimon.data.BinaryRow@c020a121, maxKey=org.apache.paimon.data.BinaryRow@bcc86d21}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-2.orc', minKey=org.apache.paimon.data.BinaryRow@60bffc9f, maxKey=org.apache.paimon.data.BinaryRow@87e819e1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-3.orc', minKey=org.apache.paimon.data.BinaryRow@f8b5b2a1, maxKey=org.apache.paimon.data.BinaryRow@831b2521}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-4.orc', minKey=org.apache.paimon.data.BinaryRow@c1a152e1, maxKey=org.apache.paimon.data.BinaryRow@b06c47e1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-5.orc', minKey=org.apache.paimon.data.BinaryRow@f40617e1, maxKey=org.apache.paimon.data.BinaryRow@69797d5f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-6.orc', minKey=org.apache.paimon.data.BinaryRow@1196e39f, maxKey=org.apache.paimon.data.BinaryRow@dde62d61}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-7.orc', minKey=org.apache.paimon.data.BinaryRow@ac5e4ca1, maxKey=org.apache.paimon.data.BinaryRow@fa627ea1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-8.orc', minKey=org.apache.paimon.data.BinaryRow@12ad64df, maxKey=org.apache.paimon.data.BinaryRow@2d67445f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-9.orc', minKey=org.apache.paimon.data.BinaryRow@a1fccfe1, maxKey=org.apache.paimon.data.BinaryRow@2e90249f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-10.orc', minKey=org.apache.paimon.data.BinaryRow@1646f5f, maxKey=org.apache.paimon.data.BinaryRow@fab41821}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-11.orc', minKey=org.apache.paimon.data.BinaryRow@8cec6b61, maxKey=org.apache.paimon.data.BinaryRow@df034df}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-12.orc', minKey=org.apache.paimon.data.BinaryRow@621adadf, maxKey=org.apache.paimon.data.BinaryRow@f8cd4661}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-13.orc', minKey=org.apache.paimon.data.BinaryRow@9bf5d2e1, maxKey=org.apache.paimon.data.BinaryRow@e9259221}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-14.orc', minKey=org.apache.paimon.data.BinaryRow@5faec55f, maxKey=org.apache.paimon.data.BinaryRow@1e49e1f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-15.orc', minKey=org.apache.paimon.data.BinaryRow@a5215be1, maxKey=org.apache.paimon.data.BinaryRow@cf1a84e1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-16.orc', minKey=org.apache.paimon.data.BinaryRow@b0b3fbe1, maxKey=org.apache.paimon.data.BinaryRow@de3cf2e1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-17.orc', minKey=org.apache.paimon.data.BinaryRow@bd4f52e1, maxKey=org.apache.paimon.data.BinaryRow@e0993a21}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-18.orc', minKey=org.apache.paimon.data.BinaryRow@a9a9b0a1, maxKey=org.apache.paimon.data.BinaryRow@f3f969f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-19.orc', minKey=org.apache.paimon.data.BinaryRow@9455da61, maxKey=org.apache.paimon.data.BinaryRow@16ebe55f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-20.orc', minKey=org.apache.paimon.data.BinaryRow@9bc8df61, maxKey=org.apache.paimon.data.BinaryRow@e4e533a1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-21.orc', minKey=org.apache.paimon.data.BinaryRow@aed529a1, maxKey=org.apache.paimon.data.BinaryRow@90a12f61}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-22.orc', minKey=org.apache.paimon.data.BinaryRow@8272c461, maxKey=org.apache.paimon.data.BinaryRow@1fd1f9df}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-23.orc', minKey=org.apache.paimon.data.BinaryRow@9808e921, maxKey=org.apache.paimon.data.BinaryRow@6a30709f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-24.orc', minKey=org.apache.paimon.data.BinaryRow@ddf851f, maxKey=org.apache.paimon.data.BinaryRow@541bf55f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-25.orc', minKey=org.apache.paimon.data.BinaryRow@34c78f9f, maxKey=org.apache.paimon.data.BinaryRow@1ce6755f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-26.orc', minKey=org.apache.paimon.data.BinaryRow@232f07df, maxKey=org.apache.paimon.data.BinaryRow@3ba7cd5f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-27.orc', minKey=org.apache.paimon.data.BinaryRow@4c77959f, maxKey=org.apache.paimon.data.BinaryRow@fb4c7721}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-28.orc', minKey=org.apache.paimon.data.BinaryRow@4918331f, maxKey=org.apache.paimon.data.BinaryRow@bd6b1e61}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-29.orc', minKey=org.apache.paimon.data.BinaryRow@ef7d8361, maxKey=org.apache.paimon.data.BinaryRow@31fcd8df}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-30.orc', minKey=org.apache.paimon.data.BinaryRow@7925549f, maxKey=org.apache.paimon.data.BinaryRow@f5a1a8a1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-31.orc', minKey=org.apache.paimon.data.BinaryRow@2b13d4df, maxKey=org.apache.paimon.data.BinaryRow@1c1cef9f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-32.orc', minKey=org.apache.paimon.data.BinaryRow@4dc6e19f, maxKey=org.apache.paimon.data.BinaryRow@1b4a4df}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-33.orc', minKey=org.apache.paimon.data.BinaryRow@ef731a21, maxKey=org.apache.paimon.data.BinaryRow@8f94f9f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-34.orc', minKey=org.apache.paimon.data.BinaryRow@29a1299f, maxKey=org.apache.paimon.data.BinaryRow@bc661461}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-35.orc', minKey=org.apache.paimon.data.BinaryRow@95ae1521, maxKey=org.apache.paimon.data.BinaryRow@29aeca1f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-36.orc', minKey=org.apache.paimon.data.BinaryRow@8f7249f, maxKey=org.apache.paimon.data.BinaryRow@3d68f49f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-37.orc', minKey=org.apache.paimon.data.BinaryRow@5981f41f, maxKey=org.apache.paimon.data.BinaryRow@9814e461}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-38.orc', minKey=org.apache.paimon.data.BinaryRow@33dcc71f, maxKey=org.apache.paimon.data.BinaryRow@4922299f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-39.orc', minKey=org.apache.paimon.data.BinaryRow@8e276d61, maxKey=org.apache.paimon.data.BinaryRow@6e7fe1f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-40.orc', minKey=org.apache.paimon.data.BinaryRow@39d502df, maxKey=org.apache.paimon.data.BinaryRow@6899b9f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-41.orc', minKey=org.apache.paimon.data.BinaryRow@48de24df, maxKey=org.apache.paimon.data.BinaryRow@5029809f}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-42.orc', minKey=org.apache.paimon.data.BinaryRow@5e952e5f, maxKey=org.apache.paimon.data.BinaryRow@b079e5a1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-43.orc', minKey=org.apache.paimon.data.BinaryRow@b9a089a1, maxKey=org.apache.paimon.data.BinaryRow@2d5fbdf}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-44.orc', minKey=org.apache.paimon.data.BinaryRow@9f822f61, maxKey=org.apache.paimon.data.BinaryRow@81047721}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-45.orc', minKey=org.apache.paimon.data.BinaryRow@9da7b821, maxKey=org.apache.paimon.data.BinaryRow@d1e2abe1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-46.orc', minKey=org.apache.paimon.data.BinaryRow@537aeedf, maxKey=org.apache.paimon.data.BinaryRow@d4c4fda1}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=31, level=5, fileName='data-eda43bdb-3bf4-48d1-8d8a-0a74dba498f0-47.orc', minKey=org.apache.paimon.data.BinaryRow@da284621, maxKey=org.apache.paimon.data.BinaryRow@c481c861}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=34, level=5, fileName='data-0fb904b5-8d0a-4cd7-8516-85c2385d1ef4-0.orc', minKey=org.apache.paimon.data.BinaryRow@4b0e9a22, maxKey=org.apache.paimon.data.BinaryRow@ebe7e41e}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=34, level=5, fileName='data-0fb904b5-8d0a-4cd7-8516-85c2385d1ef4-1.orc', minKey=org.apache.paimon.data.BinaryRow@95f2715e, maxKey=org.apache.paimon.data.BinaryRow@27ea0362}
Changes are:
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=37, level=0, fileName='data-19fd8e26-eeb3-4d61-87c8-87d8119d9aa7-233.orc', minKey=org.apache.paimon.data.BinaryRow@eefbdc9b, maxKey=org.apache.paimon.data.BinaryRow@a37b32db}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=37, level=0, fileName='data-975fb37d-56c3-4767-93d0-053ed8c0e21c-1.orc', minKey=org.apache.paimon.data.BinaryRow@5453f965, maxKey=org.apache.paimon.data.BinaryRow@3ec1f2e5}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=37, level=1, fileName='data-c009c914-0ad6-415a-8f87-62fa7faef4ef-114.orc', minKey=org.apache.paimon.data.BinaryRow@6d4c825, maxKey=org.apache.paimon.data.BinaryRow@37d4abe5}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=37, level=1, fileName='data-11c0a96a-3138-427e-aeda-73868bd84d71-0.orc', minKey=org.apache.paimon.data.BinaryRow@5453f965, maxKey=org.apache.paimon.data.BinaryRow@3ec1f2e5}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=42, level=0, fileName='data-341299c5-6126-4e97-90d7-f53532a02f7a-1421.orc', minKey=org.apache.paimon.data.BinaryRow@7c54a9ea, maxKey=org.apache.paimon.data.BinaryRow@dbbcd6a}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=42, level=0, fileName='data-87e8fabf-b9bd-4e84-a509-d31e30d2e379-1.orc', minKey=org.apache.paimon.data.BinaryRow@15072aea, maxKey=org.apache.paimon.data.BinaryRow@6b71422a}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=42, level=1, fileName='data-942ebe35-e4f3-4df2-a4b8-82e60394c9c2-698.orc', minKey=org.apache.paimon.data.BinaryRow@da7c53d6, maxKey=org.apache.paimon.data.BinaryRow@12a2afea}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=42, level=1, fileName='data-9f918459-4fc8-4991-8783-c709e7c80c8c-0.orc', minKey=org.apache.paimon.data.BinaryRow@15072aea, maxKey=org.apache.paimon.data.BinaryRow@6b71422a}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=43, level=0, fileName='data-64aa8e97-b138-4da7-8d40-2fb6887b954f-1427.orc', minKey=org.apache.paimon.data.BinaryRow@7f74906b, maxKey=org.apache.paimon.data.BinaryRow@d519ebd5}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=43, level=1, fileName='data-d517bba2-f709-4432-be95-91bfaf04143d-702.orc', minKey=org.apache.paimon.data.BinaryRow@c626cc15, maxKey=org.apache.paimon.data.BinaryRow@44375d6b}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=43, level=0, fileName='data-29d21725-040e-438d-923d-77b708dc5d0b-1.orc', minKey=org.apache.paimon.data.BinaryRow@6e9da5ab, maxKey=org.apache.paimon.data.BinaryRow@5780d4ab}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=43, level=1, fileName='data-f7c6791a-25c1-4ef2-97fc-bd06c8b8e5a7-0.orc', minKey=org.apache.paimon.data.BinaryRow@6e9da5ab, maxKey=org.apache.paimon.data.BinaryRow@44375d6b}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=52, level=1, fileName='data-a44e51a9-c1a8-4e4f-9ff5-d8f78c8b0f1a-673.orc', minKey=org.apache.paimon.data.BinaryRow@25ef0ff4, maxKey=org.apache.paimon.data.BinaryRow@279e9334}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=52, level=0, fileName='data-f15d1be7-bed5-4540-9f85-812bb9f941b7-1431.orc', minKey=org.apache.paimon.data.BinaryRow@d73964cc, maxKey=org.apache.paimon.data.BinaryRow@495888b4}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=52, level=2, fileName='data-a44e51a9-c1a8-4e4f-9ff5-d8f78c8b0f1a-672.orc', minKey=org.apache.paimon.data.BinaryRow@43fad634, maxKey=org.apache.paimon.data.BinaryRow@6ab4c0f4}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=52, level=0, fileName='data-33ba253a-ac38-45ef-a93d-83842123daa6-1.orc', minKey=org.apache.paimon.data.BinaryRow@65a0a9b4, maxKey=org.apache.paimon.data.BinaryRow@981f4f4}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=52, level=2, fileName='data-ab8e0631-56db-46dc-802f-3c649138cd13-0.orc', minKey=org.apache.paimon.data.BinaryRow@65a0a9b4, maxKey=org.apache.paimon.data.BinaryRow@981f4f4}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=24, level=0, fileName='data-4152f05b-72af-49ae-9033-4d9390577a33-1427.orc', minKey=org.apache.paimon.data.BinaryRow@b323c58, maxKey=org.apache.paimon.data.BinaryRow@164ce58}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=24, level=1, fileName='data-52079d7c-b4b2-4779-84ae-72ee3f3e5b98-676.orc', minKey=org.apache.paimon.data.BinaryRow@51616ed8, maxKey=org.apache.paimon.data.BinaryRow@39b67258}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=24, level=2, fileName='data-52079d7c-b4b2-4779-84ae-72ee3f3e5b98-661.orc', minKey=org.apache.paimon.data.BinaryRow@28c68a18, maxKey=org.apache.paimon.data.BinaryRow@657e8f98}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=24, level=0, fileName='data-5c580eda-dc07-41cd-a9a8-3a1df276e8ac-1.orc', minKey=org.apache.paimon.data.BinaryRow@3e2f7f98, maxKey=org.apache.paimon.data.BinaryRow@bcd1a028}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=24, level=2, fileName='data-049815d2-f797-483e-aaef-d7ebf9e2568f-0.orc', minKey=org.apache.paimon.data.BinaryRow@3e2f7f98, maxKey=org.apache.paimon.data.BinaryRow@bcd1a028}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=0, fileName='data-6477a9c0-2b9f-4e1a-8b75-75cfb03a77ee-1429.orc', minKey=org.apache.paimon.data.BinaryRow@6aadea33, maxKey=org.apache.paimon.data.BinaryRow@55398af3}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=0, fileName='data-53f10294-3b79-4ceb-a337-a7f10026be1d-1.orc', minKey=org.apache.paimon.data.BinaryRow@927c29cd, maxKey=org.apache.paimon.data.BinaryRow@a2cd214d}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=1, fileName='data-b51ebdeb-4cef-4b10-86fc-d70fcc282642-679.orc', minKey=org.apache.paimon.data.BinaryRow@31ea51b3, maxKey=org.apache.paimon.data.BinaryRow@a0c8840d}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=3, fileName='data-b51ebdeb-4cef-4b10-86fc-d70fcc282642-223.orc', minKey=org.apache.paimon.data.BinaryRow@13c39fb3, maxKey=org.apache.paimon.data.BinaryRow@3a0e2b33}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=2, fileName='data-b51ebdeb-4cef-4b10-86fc-d70fcc282642-446.orc', minKey=org.apache.paimon.data.BinaryRow@a067e44d, maxKey=org.apache.paimon.data.BinaryRow@5d233273}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=4, fileName='data-b51ebdeb-4cef-4b10-86fc-d70fcc282642-0.orc', minKey=org.apache.paimon.data.BinaryRow@c2af094d, maxKey=org.apache.paimon.data.BinaryRow@fae7a70d}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=51, level=4, fileName='data-72a030b2-f604-4f8e-b58a-d296a895399b-0.orc', minKey=org.apache.paimon.data.BinaryRow@31ea51b3, maxKey=org.apache.paimon.data.BinaryRow@3a0e2b33}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=53, level=0, fileName='data-d66381fe-74ae-485f-bace-d6eb6ae4c010-1431.orc', minKey=org.apache.paimon.data.BinaryRow@9a15dacb, maxKey=org.apache.paimon.data.BinaryRow@411a2ef5}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=53, level=1, fileName='data-bc06f732-a1db-487c-90fc-2923f718c404-675.orc', minKey=org.apache.paimon.data.BinaryRow@b6f5960b, maxKey=org.apache.paimon.data.BinaryRow@258b4935}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=53, level=2, fileName='data-bc06f732-a1db-487c-90fc-2923f718c404-569.orc', minKey=org.apache.paimon.data.BinaryRow@a1d6f20b, maxKey=org.apache.paimon.data.BinaryRow@e83b864b}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=53, level=0, fileName='data-dbb67efb-99cd-461e-9a10-206c2f311a25-1.orc', minKey=org.apache.paimon.data.BinaryRow@4cddbdb5, maxKey=org.apache.paimon.data.BinaryRow@507e4475}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=53, level=2, fileName='data-760fafac-0a11-4420-98cf-4b409c19691a-0.orc', minKey=org.apache.paimon.data.BinaryRow@4cddbdb5, maxKey=org.apache.paimon.data.BinaryRow@507e4475}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=60, level=0, fileName='data-39e993b3-3de3-4fd5-a2da-a08a83bb27cf-1433.orc', minKey=org.apache.paimon.data.BinaryRow@6324007c, maxKey=org.apache.paimon.data.BinaryRow@50739c3c}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=60, level=3, fileName='data-1d633c4e-7ba7-45a0-a800-5dba80a12a79-672.orc', minKey=org.apache.paimon.data.BinaryRow@a73c1984, maxKey=org.apache.paimon.data.BinaryRow@1621977c}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=60, level=2, fileName='data-1d633c4e-7ba7-45a0-a800-5dba80a12a79-678.orc', minKey=org.apache.paimon.data.BinaryRow@1282917c, maxKey=org.apache.paimon.data.BinaryRow@726d57bc}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=60, level=0, fileName='data-b3b8adb4-320f-4576-b90f-a154393b24fc-1.orc', minKey=org.apache.paimon.data.BinaryRow@a0cca2c4, maxKey=org.apache.paimon.data.BinaryRow@7ac72afc}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=60, level=3, fileName='data-ac05813a-32bf-49ee-aef1-9ae6ad7592ce-0.orc', minKey=org.apache.paimon.data.BinaryRow@a0cca2c4, maxKey=org.apache.paimon.data.BinaryRow@7ac72afc}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=48, level=0, fileName='data-8abde626-cfa9-48e7-ab18-03dcc0faabec-1431.orc', minKey=org.apache.paimon.data.BinaryRow@9d0206d0, maxKey=org.apache.paimon.data.BinaryRow@f0802b10}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=48, level=0, fileName='data-b2edcd3a-6358-4f42-9bce-0863e3c66466-1.orc', minKey=org.apache.paimon.data.BinaryRow@795b9270, maxKey=org.apache.paimon.data.BinaryRow@39b153f0}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=48, level=1, fileName='data-2121369d-785c-4282-bfa6-a64447ee5c29-705.orc', minKey=org.apache.paimon.data.BinaryRow@e97b3ed0, maxKey=org.apache.paimon.data.BinaryRow@a5d6e150}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=48, level=1, fileName='data-c758d136-64ea-421c-ae62-518259b00a2e-0.orc', minKey=org.apache.paimon.data.BinaryRow@e97b3ed0, maxKey=org.apache.paimon.data.BinaryRow@39b153f0}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=61, level=0, fileName='data-ce11c3a1-89d8-4946-a955-273bf1b0f1c9-1431.orc', minKey=org.apache.paimon.data.BinaryRow@4f43a9bd, maxKey=org.apache.paimon.data.BinaryRow@45e3067d}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=61, level=1, fileName='data-9b764057-5a10-4e6d-8a24-4290456836b8-714.orc', minKey=org.apache.paimon.data.BinaryRow@3a8d22bd, maxKey=org.apache.paimon.data.BinaryRow@c3b2c3c3}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=61, level=0, fileName='data-39c46c95-2498-447c-82f7-105f53115450-1.orc', minKey=org.apache.paimon.data.BinaryRow@964dd503, maxKey=org.apache.paimon.data.BinaryRow@dbd9d443}
{kind=ADD, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=61, level=1, fileName='data-8a857c00-90f0-43a0-9167-ac41e0b7a3d4-0.orc', minKey=org.apache.paimon.data.BinaryRow@3a8d22bd, maxKey=org.apache.paimon.data.BinaryRow@c3b2c3c3}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=59, level=0, fileName='data-0b85af4f-7c88-4c84-8faa-38b5e87144d3-1431.orc', minKey=org.apache.paimon.data.BinaryRow@4407dbbb, maxKey=org.apache.paimon.data.BinaryRow@c4a77bc5}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=59, level=0, fileName='data-e33879f0-c44d-4315-bb1f-c00cddfa12e6-1.orc', minKey=org.apache.paimon.data.BinaryRow@16f83d3b, maxKey=org.apache.paimon.data.BinaryRow@5ae6aefb}
{kind=DELETE, partition=org.apache.paimon.data.BinaryRow@9c67b85d, bucket=59, level=2, fileName='data-4c413d2d-7df2-4a31-a605-25900daf2ba1-442.orc', minKey=org.apache.paimon.data.BinaryRow@96c819c5, maxKey=org.apache.paimon.data.BinaryRow@1555d8fb}
The entry list above are not fully displayed, please refer to taskmanager.log for more information.
at org.apache.paimon.operation.FileStoreCommitImpl.createConflictException(FileStoreCommitImpl.java:1138) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:1010) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:990) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
Caused by: java.lang.IllegalStateException: Trying to delete file data-19fd8e26-eeb3-4d61-87c8-87d8119d9aa7-233.orc which is not previously added.
at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.assertNoDelete(FileEntry.java:181) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:1007) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
... 20 more
2025-08-23 08:07:08,016 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 95c2d839e2e53a32f408c6e800c6df67
2025-08-23 08:07:08,021 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - 1 tasks will be restarted to recover the failed task 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_0.
2025-08-23 08:07:08,021 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RUNNING to RESTARTING.
2025-08-23 08:07:13,030 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RESTARTING to RUNNING.
2025-08-23 08:07:13,032 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1) switched from CREATED to SCHEDULED.
2025-08-23 08:07:13,033 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 95c2d839e2e53a32f408c6e800c6df67: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}]
2025-08-23 08:07:13,033 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1) switched from SCHEDULED to DEPLOYING.
2025-08-23 08:07:13,033 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (attempt #1) with attempt id 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1 and vertex id 250bb1b6a5242fdebff09c57fe4d1696_0 to container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473) with allocation id 2691f70b7c3b3db638d93c0aede41d20
2025-08-23 08:07:13,049 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1) switched from DEPLOYING to INITIALIZING.
2025-08-23 08:07:14,575 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1) switched from INITIALIZING to RUNNING.
2025-08-23 08:07:15,318 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1) switched from RUNNING to FAILED on container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473).
java.lang.RuntimeException: Cannot read manifest entries from changed partitions.
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:984) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:992) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
Caused by: java.lang.IllegalStateException: Trying to add file {org.apache.paimon.data.BinaryRow@9c67b85d, 37, 0, data-975fb37d-56c3-4767-93d0-053ed8c0e21c-1.orc} which is already added.
at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:134) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:114) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readAndMergeFileEntries(AbstractFileStoreScan.java:397) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readSimpleEntries(AbstractFileStoreScan.java:259) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:982) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
... 20 more
2025-08-23 08:07:15,320 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 95c2d839e2e53a32f408c6e800c6df67
2025-08-23 08:07:15,321 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - 1 tasks will be restarted to recover the failed task 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_1.
2025-08-23 08:07:15,321 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RUNNING to RESTARTING.
2025-08-23 08:07:20,322 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RESTARTING to RUNNING.
2025-08-23 08:07:20,322 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2) switched from CREATED to SCHEDULED.
2025-08-23 08:07:20,323 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 95c2d839e2e53a32f408c6e800c6df67: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}]
2025-08-23 08:07:20,323 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2) switched from SCHEDULED to DEPLOYING.
2025-08-23 08:07:20,323 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (attempt #2) with attempt id 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2 and vertex id 250bb1b6a5242fdebff09c57fe4d1696_0 to container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473) with allocation id 2691f70b7c3b3db638d93c0aede41d20
2025-08-23 08:07:20,335 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2) switched from DEPLOYING to INITIALIZING.
2025-08-23 08:07:21,252 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2) switched from INITIALIZING to RUNNING.
2025-08-23 08:07:22,321 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2) switched from RUNNING to FAILED on container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473).
java.lang.RuntimeException: Cannot read manifest entries from changed partitions.
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:984) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:992) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
Caused by: java.lang.IllegalStateException: Trying to add file {org.apache.paimon.data.BinaryRow@9c67b85d, 37, 0, data-975fb37d-56c3-4767-93d0-053ed8c0e21c-1.orc} which is already added.
at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:134) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:114) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readAndMergeFileEntries(AbstractFileStoreScan.java:397) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readSimpleEntries(AbstractFileStoreScan.java:259) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:982) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
... 20 more
2025-08-23 08:07:22,322 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 95c2d839e2e53a32f408c6e800c6df67
2025-08-23 08:07:22,323 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - 1 tasks will be restarted to recover the failed task 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_2.
2025-08-23 08:07:22,323 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RUNNING to RESTARTING.
2025-08-23 08:07:27,324 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RESTARTING to RUNNING.
2025-08-23 08:07:27,324 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3) switched from CREATED to SCHEDULED.
2025-08-23 08:07:27,325 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3) switched from SCHEDULED to DEPLOYING.
2025-08-23 08:07:27,324 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 95c2d839e2e53a32f408c6e800c6df67: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}]
2025-08-23 08:07:27,325 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (attempt #3) with attempt id 4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3 and vertex id 250bb1b6a5242fdebff09c57fe4d1696_0 to container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473) with allocation id 2691f70b7c3b3db638d93c0aede41d20
2025-08-23 08:07:27,338 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3) switched from DEPLOYING to INITIALIZING.
2025-08-23 08:07:28,229 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3) switched from INITIALIZING to RUNNING.
2025-08-23 08:07:29,558 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Global Committer : yydw_dm_entity_person_paimon_mini -> Sink: end (1/1) (4460de05e51badec3bb31e202627eb02_250bb1b6a5242fdebff09c57fe4d1696_0_3) switched from RUNNING to FAILED on container_e256_1755049813097_645803_01_000002 @ fs-hiido-dn-12-68-86.hiido.host.int.yy.com (dataPort=39473).
java.lang.RuntimeException: Cannot read manifest entries from changed partitions.
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:984) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:992) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
Caused by: java.lang.IllegalStateException: Trying to add file {org.apache.paimon.data.BinaryRow@9c67b85d, 37, 0, data-975fb37d-56c3-4767-93d0-053ed8c0e21c-1.orc} which is already added.
at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:134) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:114) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readAndMergeFileEntries(AbstractFileStoreScan.java:397) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readSimpleEntries(AbstractFileStoreScan.java:259) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:982) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
... 20 more
2025-08-23 08:07:29,560 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 95c2d839e2e53a32f408c6e800c6df67
2025-08-23 08:07:29,561 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job kyuubi_10_12_36_75_20250823080032423_759163 (95c2d839e2e53a32f408c6e800c6df67) switched from state RUNNING to FAILING.
org.apache.flink.runtime.JobException: Recovery is suppressed by FixedDelayRestartBackoffTimeStrategy(maxNumberRestartAttempts=3, backoffTimeMS=5000)
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:139) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:83) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.DefaultScheduler.recordTaskFailure(DefaultScheduler.java:258) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:249) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.DefaultScheduler.onTaskFailed(DefaultScheduler.java:242) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.SchedulerBase.onTaskExecutionStateUpdate(SchedulerBase.java:748) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:725) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:80) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:479) ~[flink-dist-1.17.2.jar:1.17.2]
at sun.reflect.GeneratedMethodAccessor79.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_311]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_311]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:309) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:307) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:222) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:84) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:168) ~[flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.actor.Actor.aroundReceive(Actor.scala:537) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.actor.Actor.aroundReceive$(Actor.scala:535) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.actor.ActorCell.invoke(ActorCell.scala:547) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.dispatch.Mailbox.run(Mailbox.scala:231) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [flink-rpc-akka_2e32811a-6cc7-4984-b67d-7f936342da47.jar:1.17.2]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) [?:1.8.0_311]
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1067) [?:1.8.0_311]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1703) [?:1.8.0_311]
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:172) [?:1.8.0_311]
Caused by: java.lang.RuntimeException: Cannot read manifest entries from changed partitions.
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:984) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:992) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
Caused by: java.lang.IllegalStateException: Trying to add file {org.apache.paimon.data.BinaryRow@9c67b85d, 37, 0, data-975fb37d-56c3-4767-93d0-053ed8c0e21c-1.orc} which is already added.
at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:134) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.manifest.FileEntry.mergeEntries(FileEntry.java:114) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readAndMergeFileEntries(AbstractFileStoreScan.java:397) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.AbstractFileStoreScan.readSimpleEntries(AbstractFileStoreScan.java:259) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.readAllEntriesFromChangedPartitions(FileStoreCommitImpl.java:982) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:992) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommitOnce(FileStoreCommitImpl.java:773) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.tryCommit(FileStoreCommitImpl.java:659) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:302) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:206) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:100) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:171) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.paimon.flink.sink.CommitterOperator.endInput(CommitterOperator.java:158) ~[paimon-flink-1.17-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.endOperatorInput(StreamOperatorWrapper.java:96) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.endInput(RegularOperatorChain.java:97) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:68) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:550) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_311]
2025-08-23 08:07:29,566 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 4460de05e51badec3bb31e202627eb02_cbc357ccb763df2852fee8c4fc7d55f2_0_0.
2025-08-23 08:07:29,567 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 4460de05e51badec3bb31e202627eb02_cbc357ccb763df2852fee8c4fc7d55f2_1_0.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]Flink deletes data error in paimon [paimon]
Posted by "ASiegeLion (via GitHub)" <gi...@apache.org>.
ASiegeLion commented on issue #6137:
URL: https://github.com/apache/paimon/issues/6137#issuecomment-3219644476
<img width="1495" height="430" alt="Image" src="https://github.com/user-attachments/assets/510b2d74-65b6-42bf-b84f-abd0c22cdca7" />
flink batch sql ： delete from not supported set 'write-only'='true' ？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]Flink deletes data error in paimon [paimon]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on issue #6137:
URL: https://github.com/apache/paimon/issues/6137#issuecomment-3223655132
Can you pls explain the expectation more clearly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]Flink deletes data error in paimon [paimon]
Posted by "ASiegeLion (via GitHub)" <gi...@apache.org>.
ASiegeLion commented on issue #6137:
URL: https://github.com/apache/paimon/issues/6137#issuecomment-3224411069
thanks, I have two questions：
1 ，Why doesn't DeleteAction support the write-only=true options?
2 , flink batch sql:
`delete from paimon.hainadw.yydw_dm_entity_person_paimon_mini/*+ OPTIONS('write-only'='true') */
where last_update_dt < TIMESTAMPADD(DAY, -366, CURRENT_DATE);`
This write-only seems to only work on scanTable, not sink table, because when I ran Flink's delete unit test, I found that the sink table could not get options.
<img width="1197" height="117" alt="Image" src="https://github.com/user-attachments/assets/5b40b4c9-14cb-4c40-ab15-e4da0b80faae" />
<img width="995" height="575" alt="Image" src="https://github.com/user-attachments/assets/cabb3fbb-3420-4ab8-b295-02121e7bca1d" />
From the above, we can see that LogicalSink did not get options, only LogicalTableScan got options. This seems to be a problem with the Flink delete from sql parsing hint?
> Can you pls explain the expectation more clearly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]Flink deletes data error in paimon [paimon]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on issue #6137:
URL: https://github.com/apache/paimon/issues/6137#issuecomment-3223651920
@ASiegeLion IIUC this is the expected behaviour, DELETE shouldn’t be run with write-only to true cc @JingsongLi. If you are running multiple jobs at the same time and conflict resolution is impossible, paimon will give up causing this error.
That being said we can fail fast, instead of letting this go to the committer if write-only is enabled with delete. I'll raise a patch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org