You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "AlexNilone (via GitHub)" <gi...@apache.org> on 2023/08/23 02:12:56 UTC
[GitHub] [incubator-paimon] AlexNilone opened a new issue, #1871: [Bug] The tag feature stopped working
AlexNilone opened a new issue, #1871:
URL: https://github.com/apache/incubator-paimon/issues/1871
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Paimon version
paimon-flink-1.17-0.5-20230808.002215-115.jar
### Compute Engine
Flink1.17.1
Spark3.3
### Minimal reproduce step
imulate an abnormal interruption of the Flink streaming job, where the process exists but is unable to write to Paimon.
### What doesn't meet your expectations?
I've been using the tagging feature, but it's not performing very reliably. Whenever my Flink data lake job is abnormally interrupted (due to issues like insufficient memory or network problems), and I restart the job, I find that the tags are no longer being generated. This issue has occurred twice. The first time, there were some exceptions in Flink, while the second time there were no exceptions at all.
SELECT * FROM paimon.abc.`xxxxxxxxxxxx$tags` ;
+----------------+--------------+------------+--------------------------+---------------+
| tag_name | snapshot_id | schema_id | commit_time | record_count |
+----------------+--------------+------------+--------------------------+---------------+
| 2023-08-15 21 | 12896 | 0 | 2023-08-15 22:05:17.967 | 36636402015 |
| 2023-08-15 22 | 13016 | 0 | 2023-08-15 23:05:18.0 | 37539513865 |
| 2023-08-15 23 | 13136 | 0 | 2023-08-16 00:05:17.944 | 38553088846 |
| 2023-08-16 00 | 13256 | 0 | 2023-08-16 01:05:17.918 | 39481076700 |
| 2023-08-16 01 | 13376 | 0 | 2023-08-16 02:05:17.942 | 40624835369 |
| 2023-08-16 02 | 13496 | 0 | 2023-08-16 03:05:17.927 | 42175561025 |
| 2023-08-16 03 | 13616 | 0 | 2023-08-16 04:05:17.928 | 42820694449 |
| 2023-08-16 04 | 13736 | 0 | 2023-08-16 05:05:17.966 | 43368545506 |
| 2023-08-16 05 | 13856 | 0 | 2023-08-16 06:05:17.969 | 44062803728 |
| 2023-08-16 06 | 13976 | 0 | 2023-08-16 07:05:17.955 | 45004113384 |
| 2023-08-16 07 | 14096 | 0 | 2023-08-16 08:05:18.196 | 46222116260 |
| 2023-08-16 08 | 14216 | 0 | 2023-08-16 09:05:17.923 | 47385053423 |
| 2023-08-16 09 | 14336 | 0 | 2023-08-16 10:05:17.999 | 47997117332 |
+----------------+--------------+------------+--------------------------+---------------+
13 rows selected (0.284 seconds)
the first time exception:
2023-08-11 09:01:06
java.lang.RuntimeException: File deletion conflicts detected! Give up committing.
Don't panic!
Conflicts during commits are normal and this failure is intended to resolve the conflicts.
Conflicts are mainly caused by the following scenarios:
1. Your job is suffering from back-pressuring.
There are too many snapshots waiting to be committed and an exception occurred during the commit procedure (most probably due to checkpoint timeout).
See https://paimon.apache.org/docs/master/maintenance/write-performance/ for how to improve writing performance.
2. Multiple jobs are writing into the same partition at the same time (you'll probably see different base commit user and current commit user below).
You can use https://paimon.apache.org/docs/master/maintenance/multiple-writers/#dedicated-compaction-job to support multiple writing.
3. You're recovering from an old savepoint, or you're creating multiple jobs from a savepoint.
The job will fail continuously in this scenario to protect metadata from corruption.
You can either recover from the latest savepoint, or you can revert the table to the snapshot corresponding to the old savepoint.
Base commit user is: 9b39c993-9cd4-412f-bf1f-38e8699d82d2; Current commit user is: a176e41e-4a55-4921-b0d0-19ef769e5b15
Base entries are:
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-5582a43d-3870-4f30-93c6-310446de8481-514.orc, 180051589, 8849408, org.apache.paimon.data.BinaryRow@d2c4b35, org.apache.paimon.data.BinaryRow@63c6233f, org.apache.paimon.stats.BinaryTableStats@88450b14, org.apache.paimon.stats.BinaryTableStats@1a49bb58, 1, 13697219, 0, 5, [], 2023-08-10T10:19:23.366}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-5582a43d-3870-4f30-93c6-310446de8481-515.orc, 80450397, 3949508, org.apache.paimon.data.BinaryRow@e24fa137, org.apache.paimon.data.BinaryRow@5011d334, org.apache.paimon.stats.BinaryTableStats@3f153741, org.apache.paimon.stats.BinaryTableStats@f97a9d21, 0, 13697220, 0, 5, [], 2023-08-10T10:19:30.175}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-c987f986-91f8-42a6-ad88-7b7b694dfce2-518.orc, 175454714, 8650752, org.apache.paimon.data.BinaryRow@86fe5854, org.apache.paimon.data.BinaryRow@e843106, org.apache.paimon.stats.BinaryTableStats@82cbf78c, org.apache.paimon.stats.BinaryTableStats@253e663d, 0, 14040581, 0, 5, [], 2023-08-10T10:51:24.413}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-c987f986-91f8-42a6-ad88-7b7b694dfce2-519.orc, 90217054, 4449499, org.apache.paimon.data.BinaryRow@80b5fd26, org.apache.paimon.data.BinaryRow@d981dce0, org.apache.paimon.stats.BinaryTableStats@81e57f44, org.apache.paimon.stats.BinaryTableStats@5733e6b1, 1, 14040579, 0, 5, [], 2023-08-10T10:51:31.551}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-123.orc, 174346111, 8807424, org.apache.paimon.data.BinaryRow@b39f8d55, org.apache.paimon.data.BinaryRow@d9419d91, org.apache.paimon.stats.BinaryTableStats@98e41522, org.apache.paimon.stats.BinaryTableStats@2daedf07, 0, 17019070, 0, 5, [], 2023-08-10T20:51:24.483}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-124.orc, 135467398, 6841807, org.apache.paimon.data.BinaryRow@70d0aa93, org.apache.paimon.data.BinaryRow@4138488b, org.apache.paimon.stats.BinaryTableStats@65218f26, org.apache.paimon.stats.BinaryTableStats@f4389763, 1, 17019069, 0, 5, [], 2023-08-10T20:51:38.485}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-180.orc, 172326946, 8747008, org.apache.paimon.data.BinaryRow@199929e6, org.apache.paimon.data.BinaryRow@f5cd6583, org.apache.paimon.stats.BinaryTableStats@dbd607c1, org.apache.paimon.stats.BinaryTableStats@1c3882f7, 2, 18457671, 0, 5, [], 2023-08-10T23:07:24.700}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-181.orc, 159692618, 8108901, org.apache.paimon.data.BinaryRow@e6da0036, org.apache.paimon.data.BinaryRow@c67bd9f9, org.apache.paimon.stats.BinaryTableStats@a15aa45b, org.apache.paimon.stats.BinaryTableStats@fde7ca53, 0, 18457672, 0, 5, [], 2023-08-10T23:07:38.272}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-205.orc, 171848395, 8738816, org.apache.paimon.data.BinaryRow@1e1b0b9b, org.apache.paimon.data.BinaryRow@1f3debce, org.apache.paimon.stats.BinaryTableStats@cc06934b, org.apache.paimon.stats.BinaryTableStats@cd7797a6, 1, 19151876, 0, 5, [], 2023-08-11T00:11:22.453}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-206.orc, 170942852, 8692606, org.apache.paimon.data.BinaryRow@cdac58e4, org.apache.paimon.data.BinaryRow@f522056d, org.apache.paimon.stats.BinaryTableStats@c318cc95, org.apache.paimon.stats.BinaryTableStats@ecb79baa, 0, 19151874, 0, 5, [], 2023-08-11T00:11:37.965}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-219.orc, 62288354, 2891042, org.apache.paimon.data.BinaryRow@5e67a9b0, org.apache.paimon.data.BinaryRow@5123171c, org.apache.paimon.stats.BinaryTableStats@36643e92, org.apache.paimon.stats.BinaryTableStats@1c9cd677, 17019071, 19952743, 0, 4, [], 2023-08-11T01:27:11.454}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-322.orc, 72600636, 3368832, org.apache.paimon.data.BinaryRow@f2161a1c, org.apache.paimon.data.BinaryRow@14a52808, org.apache.paimon.stats.BinaryTableStats@44f85092, org.apache.paimon.stats.BinaryTableStats@9f2c0e51, 19151877, 22578745, 0, 4, [], 2023-08-11T05:33:17.697}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-324.orc, 57465732, 2666599, org.apache.paimon.data.BinaryRow@c05a5f43, org.apache.paimon.data.BinaryRow@83794990, org.apache.paimon.stats.BinaryTableStats@feeff771, org.apache.paimon.stats.BinaryTableStats@11bbd5b, 19952744, 22655752, 0, 3, [], 2023-08-11T05:41:11.998}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-343.orc, 93614604, 4344620, org.apache.paimon.data.BinaryRow@b0605997, org.apache.paimon.data.BinaryRow@178dfcd0, org.apache.paimon.stats.BinaryTableStats@f3e26185, org.apache.paimon.stats.BinaryTableStats@63c0f08d, 18457673, 22899494, 0, 4, [], 2023-08-11T06:05:16.414}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-076dd7b3-ae69-4d99-beeb-65a47c59f986-346.orc, 174368315, 8167424, org.apache.paimon.data.BinaryRow@3932de84, org.apache.paimon.data.BinaryRow@8b6100c7, org.apache.paimon.stats.BinaryTableStats@98b4da1b, org.apache.paimon.stats.BinaryTableStats@89cb8e85, 14040582, 23081415, 0, 4, [], 2023-08-11T06:21:20.511}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-076dd7b3-ae69-4d99-beeb-65a47c59f986-347.orc, 10176563, 476535, org.apache.paimon.data.BinaryRow@a2ae5bad, org.apache.paimon.data.BinaryRow@c7a702a7, org.apache.paimon.stats.BinaryTableStats@ddbfeb24, org.apache.paimon.stats.BinaryTableStats@2d3d4456, 14040585, 23081410, 0, 4, [], 2023-08-11T06:21:21.439}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-360.orc, 23641164, 1097223, org.apache.paimon.data.BinaryRow@a7b9ada3, org.apache.paimon.data.BinaryRow@73e096f4, org.apache.paimon.stats.BinaryTableStats@a8378ced, org.apache.paimon.stats.BinaryTableStats@bcc39e21, 22578746, 23681882, 0, 3, [], 2023-08-11T07:17:10.749}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-381.orc, 23244694, 1078705, org.apache.paimon.data.BinaryRow@1094b21b, org.apache.paimon.data.BinaryRow@ddcfd72c, org.apache.paimon.stats.BinaryTableStats@1a5c1a2d, org.apache.paimon.stats.BinaryTableStats@d848db1f, 22899495, 23984022, 0, 3, [], 2023-08-11T07:47:11.655}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-382.orc, 29067300, 1348770, org.apache.paimon.data.BinaryRow@f159a293, org.apache.paimon.data.BinaryRow@30463f3d, org.apache.paimon.stats.BinaryTableStats@d9fd66b4, org.apache.paimon.stats.BinaryTableStats@eb70664e, 22655753, 24013786, 0, 2, [], 2023-08-11T07:49:10.431}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-076dd7b3-ae69-4d99-beeb-65a47c59f986-393.orc, 28183902, 1308174, org.apache.paimon.data.BinaryRow@cd0ca49a, org.apache.paimon.data.BinaryRow@a6bb7064, org.apache.paimon.stats.BinaryTableStats@ed28f6b4, org.apache.paimon.stats.BinaryTableStats@3fcf5963, 23081416, 24398362, 0, 3, [], 2023-08-11T08:25:10.683}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-419.orc, 178977369, 8482816, org.apache.paimon.data.BinaryRow@8a60c0a5, org.apache.paimon.data.BinaryRow@635270bb, org.apache.paimon.stats.BinaryTableStats@7c2f4688, org.apache.paimon.stats.BinaryTableStats@ea8383c3, 13697221, 24599968, 0, 4, [], 2023-08-11T08:45:21.171}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-420.orc, 38963984, 1847112, org.apache.paimon.data.BinaryRow@bf45907f, org.apache.paimon.data.BinaryRow@3f07f1bd, org.apache.paimon.stats.BinaryTableStats@a61a2720, org.apache.paimon.stats.BinaryTableStats@11959bf8, 13697252, 24599960, 0, 4, [], 2023-08-11T08:45:23.995}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-408.orc, 14576280, 676332, org.apache.paimon.data.BinaryRow@92d8656, org.apache.paimon.data.BinaryRow@7651ccf6, org.apache.paimon.stats.BinaryTableStats@c7cd8f1e, org.apache.paimon.stats.BinaryTableStats@db6e980c, 23984023, 24662692, 0, 2, [], 2023-08-11T08:51:09.483}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-402.orc, 21860610, 1014167, org.apache.paimon.data.BinaryRow@27c00962, org.apache.paimon.data.BinaryRow@ac7f6e2c, org.apache.paimon.stats.BinaryTableStats@1b5204b4, org.apache.paimon.stats.BinaryTableStats@3b117eec, 23681883, 24701213, 0, 2, [], 2023-08-11T08:53:08.950}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-5d54d730-79b9-4efb-8093-e3fdea465ba3-489.orc, 492362, 20879, org.apache.paimon.data.BinaryRow@7f0f2764, org.apache.paimon.data.BinaryRow@d0131a8e, org.apache.paimon.stats.BinaryTableStats@2a338a14, org.apache.paimon.stats.BinaryTableStats@95223b7d, 24738139, 24759019, 0, 0, [], 2023-08-11T08:59:05.346}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-55f6ceba-ed82-4891-95b6-3a13023da2b8-489.orc, 499414, 21114, org.apache.paimon.data.BinaryRow@1d5216e5, org.apache.paimon.data.BinaryRow@7a72fe16, org.apache.paimon.stats.BinaryTableStats@e5152acd, org.apache.paimon.stats.BinaryTableStats@dfd9eb9e, 24736086, 24757201, 0, 0, [], 2023-08-11T08:59:05.509}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-93f4f03f-3fc4-41bb-8372-43ba6501b1e0-489.orc, 504137, 21400, org.apache.paimon.data.BinaryRow@703fd4a6, org.apache.paimon.data.BinaryRow@7d21019, org.apache.paimon.stats.BinaryTableStats@520daaab, org.apache.paimon.stats.BinaryTableStats@8a2957b7, 24743734, 24765134, 0, 0, [], 2023-08-11T08:59:05.637}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-e9caee58-39aa-4839-9120-75c65b124320-489.orc, 498711, 21100, org.apache.paimon.data.BinaryRow@aa783978, org.apache.paimon.data.BinaryRow@a9a19832, org.apache.paimon.stats.BinaryTableStats@77e19e04, org.apache.paimon.stats.BinaryTableStats@8b532fe1, 24726098, 24747197, 0, 0, [], 2023-08-11T08:59:05.893}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-076dd7b3-ae69-4d99-beeb-65a47c59f986-405.orc, 7303938, 339181, org.apache.paimon.data.BinaryRow@ccaf8706, org.apache.paimon.data.BinaryRow@b4428970, org.apache.paimon.stats.BinaryTableStats@32f8f694, org.apache.paimon.stats.BinaryTableStats@ca8ec371, 24398363, 24738138, 0, 2, [], 2023-08-11T08:57:06.113}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-415.orc, 15517915, 719797, org.apache.paimon.data.BinaryRow@eb572f8c, org.apache.paimon.data.BinaryRow@80802704, org.apache.paimon.stats.BinaryTableStats@1ceaa86, org.apache.paimon.stats.BinaryTableStats@9c839572, 24013787, 24736085, 0, 1, [], 2023-08-11T08:57:07.427}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-403.orc, 988705, 42506, org.apache.paimon.data.BinaryRow@c0127f74, org.apache.paimon.data.BinaryRow@c8eb4e78, org.apache.paimon.stats.BinaryTableStats@59ef677a, org.apache.paimon.stats.BinaryTableStats@83624f2c, 24701214, 24743733, 0, 1, [], 2023-08-11T08:57:07.656}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-410.orc, 1383313, 63384, org.apache.paimon.data.BinaryRow@c9342386, org.apache.paimon.data.BinaryRow@3a79f52d, org.apache.paimon.stats.BinaryTableStats@617e7ef7, org.apache.paimon.stats.BinaryTableStats@e9fd9ce, 24662693, 24726097, 0, 1, [], 2023-08-11T08:57:07.927}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-bb73fcd6-0d4d-4ea5-b618-95dadc05c972-0.orc, 3195014, 147950, org.apache.paimon.data.BinaryRow@8a60c0a5, org.apache.paimon.data.BinaryRow@e37e895b, org.apache.paimon.stats.BinaryTableStats@18641e8, org.apache.paimon.stats.BinaryTableStats@264203bc, 24599969, 24748028, 0, 3, [], 2023-08-11T09:00:14.188}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-eaa72aea-418b-4d76-b8cd-44bccc78b8d5-490.orc, 503817, 21367, org.apache.paimon.data.BinaryRow@566026b4, org.apache.paimon.data.BinaryRow@e80a9a3c, org.apache.paimon.stats.BinaryTableStats@583a6b76, org.apache.paimon.stats.BinaryTableStats@68c5329e, 24748028, 24769397, 0, 0, [], 2023-08-11T09:01:05.213}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-5d54d730-79b9-4efb-8093-e3fdea465ba3-490.orc, 513005, 21747, org.apache.paimon.data.BinaryRow@8ef8aa46, org.apache.paimon.data.BinaryRow@42974672, org.apache.paimon.stats.BinaryTableStats@c3c92d12, org.apache.paimon.stats.BinaryTableStats@70227ad7, 24759020, 24780771, 0, 0, [], 2023-08-11T09:01:05.348}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-55f6ceba-ed82-4891-95b6-3a13023da2b8-490.orc, 502380, 21313, org.apache.paimon.data.BinaryRow@1e796802, org.apache.paimon.data.BinaryRow@5a9e0d42, org.apache.paimon.stats.BinaryTableStats@5ee37efe, org.apache.paimon.stats.BinaryTableStats@d74c349a, 24757202, 24778517, 0, 0, [], 2023-08-11T09:01:05.488}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-93f4f03f-3fc4-41bb-8372-43ba6501b1e0-490.orc, 508241, 21508, org.apache.paimon.data.BinaryRow@6a253a66, org.apache.paimon.data.BinaryRow@23d9e453, org.apache.paimon.stats.BinaryTableStats@cd235371, org.apache.paimon.stats.BinaryTableStats@dde07857, 24765135, 24786647, 0, 0, [], 2023-08-11T09:01:05.638}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-e9caee58-39aa-4839-9120-75c65b124320-490.orc, 508282, 21516, org.apache.paimon.data.BinaryRow@1fc52ccf, org.apache.paimon.data.BinaryRow@c4962275, org.apache.paimon.stats.BinaryTableStats@115bd5b8, org.apache.paimon.stats.BinaryTableStats@a1846cfa, 24747198, 24768716, 0, 0, [], 2023-08-11T09:01:05.770}}
Changes are:
{DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-421.orc, 1397321, 63959, org.apache.paimon.data.BinaryRow@afcf919a, org.apache.paimon.data.BinaryRow@f819d73f, org.apache.paimon.stats.BinaryTableStats@5531839, org.apache.paimon.stats.BinaryTableStats@6fab9030, 24599969, 24663951, 0, 3, [], 2023-08-11T08:51:05.452}}
{DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-422.orc, 971727, 41848, org.apache.paimon.data.BinaryRow@7c2d676c, org.apache.paimon.data.BinaryRow@27fc23fe, org.apache.paimon.stats.BinaryTableStats@fdfa0cac, org.apache.paimon.stats.BinaryTableStats@f86f450e, 24663952, 24705809, 0, 2, [], 2023-08-11T08:55:05.418}}
{DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-eaa72aea-418b-4d76-b8cd-44bccc78b8d5-488.orc, 498797, 21186, org.apache.paimon.data.BinaryRow@4aff31d4, org.apache.paimon.data.BinaryRow@4c4c70fb, org.apache.paimon.stats.BinaryTableStats@c53c2fb7, org.apache.paimon.stats.BinaryTableStats@f5d33dc2, 24705810, 24726995, 0, 0, [], 2023-08-11T08:57:05.212}}
{DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-eaa72aea-418b-4d76-b8cd-44bccc78b8d5-489.orc, 497459, 21030, org.apache.paimon.data.BinaryRow@43275b62, org.apache.paimon.data.BinaryRow@e37e895b, org.apache.paimon.stats.BinaryTableStats@a3112165, org.apache.paimon.stats.BinaryTableStats@8d65844, 24726996, 24748027, 0, 0, [], 2023-08-11T08:59:05.210}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-423.orc, 3194974, 147949, org.apache.paimon.data.BinaryRow@afcf919a, org.apache.paimon.data.BinaryRow@e37e895b, org.apache.paimon.stats.BinaryTableStats@8684a99d, org.apache.paimon.stats.BinaryTableStats@1242ff5d, 24599969, 24748027, 0, 3, [], 2023-08-11T08:59:05.921}}
{DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-55f6ceba-ed82-4891-95b6-3a13023da2b8-489.orc, 499414, 21114, org.apache.paimon.data.BinaryRow@1d5216e5, org.apache.paimon.data.BinaryRow@7a72fe16, org.apache.paimon.stats.BinaryTableStats@e5152acd, org.apache.paimon.stats.BinaryTableStats@dfd9eb9e, 24736086, 24757201, 0, 0, [], 2023-08-11T08:59:05.509}}
{DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-415.orc, 15517915, 719797, org.apache.paimon.data.BinaryRow@eb572f8c, org.apache.paimon.data.BinaryRow@80802704, org.apache.paimon.stats.BinaryTableStats@1ceaa86, org.apache.paimon.stats.BinaryTableStats@9c839572, 24013787, 24736085, 0, 1, [], 2023-08-11T08:57:07.427}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-416.orc, 15971002, 740757, org.apache.paimon.data.BinaryRow@eb572f8c, org.apache.paimon.data.BinaryRow@80802704, org.apache.paimon.stats.BinaryTableStats@1ceaa86, org.apache.paimon.stats.BinaryTableStats@9c839572, 24013787, 24757201, 0, 1, [], 2023-08-11T08:59:07.324}}
{DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-93f4f03f-3fc4-41bb-8372-43ba6501b1e0-489.orc, 504137, 21400, org.apache.paimon.data.BinaryRow@703fd4a6, org.apache.paimon.data.BinaryRow@7d21019, org.apache.paimon.stats.BinaryTableStats@520daaab, org.apache.paimon.stats.BinaryTableStats@8a2957b7, 24743734, 24765134, 0, 0, [], 2023-08-11T08:59:05.637}}
{DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-403.orc, 988705, 42506, org.apache.paimon.data.BinaryRow@c0127f74, org.apache.paimon.data.BinaryRow@c8eb4e78, org.apache.paimon.stats.BinaryTableStats@59ef677a, org.apache.paimon.stats.BinaryTableStats@83624f2c, 24701214, 24743733, 0, 1, [], 2023-08-11T08:57:07.656}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-404.orc, 1396832, 63894, org.apache.paimon.data.BinaryRow@c0127f74, org.apache.paimon.data.BinaryRow@c8eb4e78, org.apache.paimon.stats.BinaryTableStats@59ef677a, org.apache.paimon.stats.BinaryTableStats@77d32333, 24701214, 24765134, 0, 1, [], 2023-08-11T08:59:07.643}}
{DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-e9caee58-39aa-4839-9120-75c65b124320-489.orc, 498711, 21100, org.apache.paimon.data.BinaryRow@aa783978, org.apache.paimon.data.BinaryRow@a9a19832, org.apache.paimon.stats.BinaryTableStats@77e19e04, org.apache.paimon.stats.BinaryTableStats@8b532fe1, 24726098, 24747197, 0, 0, [], 2023-08-11T08:59:05.893}}
{DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-410.orc, 1383313, 63384, org.apache.paimon.data.BinaryRow@c9342386, org.apache.paimon.data.BinaryRow@3a79f52d, org.apache.paimon.stats.BinaryTableStats@617e7ef7, org.apache.paimon.stats.BinaryTableStats@e9fd9ce, 24662693, 24726097, 0, 1, [], 2023-08-11T08:57:07.927}}
{ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-411.orc, 1825478, 84474, org.apache.paimon.data.BinaryRow@aa783978, org.apache.paimon.data.BinaryRow@3a79f52d, org.apache.paimon.stats.BinaryTableStats@214e069, org.apache.paimon.stats.BinaryTableStats@b303e251, 24662693, 24747197, 0, 1, [], 2023-08-11T08:59:07.990}}
at org.apache.paimon.operation.FileStoreCommitImpl.createConflictException(FileStoreCommitImpl.java:901)
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:813)
at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:253)
at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:145)
at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:65)
at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:157)
at org.apache.paimon.flink.sink.CommitterOperator.notifyCheckpointComplete(CommitterOperator.java:151)
at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.notifyCheckpointComplete(StreamOperatorWrapper.java:104)
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.notifyCheckpointComplete(RegularOperatorChain.java:145)
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.notifyCheckpoint(SubtaskCheckpointCoordinatorImpl.java:467)
at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.notifyCheckpointComplete(SubtaskCheckpointCoordinatorImpl.java:400)
at org.apache.flink.streaming.runtime.tasks.StreamTask.notifyCheckpointComplete(StreamTask.java:1430)
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointCompleteAsync$16(StreamTask.java:1371)
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointOperation$19(StreamTask.java:1410)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:367)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:352)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Trying to delete file data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-421.orc which is not previously added. Manifest might be corrupted.
at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204)
at org.apache.paimon.manifest.ManifestEntry.assertNoDelete(ManifestEntry.java:154)
at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:810)
... 25 more
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] The tag feature stopped working [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi closed issue #1871: [Bug] The tag feature stopped working
URL: https://github.com/apache/paimon/issues/1871
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] zhuangchong commented on issue #1871: [Bug] The tag feature stopped working
Posted by "zhuangchong (via GitHub)" <gi...@apache.org>.
zhuangchong commented on issue #1871:
URL: https://github.com/apache/incubator-paimon/issues/1871#issuecomment-1689213451
@AlexNilone This PR #1810 may solve your problem, can you use the latest jar package and test it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] AlexNilone commented on issue #1871: [Bug] The tag feature stopped working
Posted by "AlexNilone (via GitHub)" <gi...@apache.org>.
AlexNilone commented on issue #1871:
URL: https://github.com/apache/incubator-paimon/issues/1871#issuecomment-1692838079
ok.let me have a test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org