You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "bridgeDream (via GitHub)" <gi...@apache.org> on 2023/11/07 05:37:11 UTC
[I] [Bug] Streaming writes to a Deduplicate table, flink job can't recover for FileNotFoundException [incubator-paimon]
bridgeDream opened a new issue, #2273:
URL: https://github.com/apache/incubator-paimon/issues/2273
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Paimon version
0.6-SNAPSHOT
### Compute Engine
Flink 1.16.1
### Minimal reproduce step
1. DDL
`CREATE TABLE IF NOT EXISTS paimon_catalog.dim.event_log_aggregation(
order_no string,
new_status int,
creator string ,
id bigint,
create_time timestamp COMMENT '日志创建时间',
version bigint COMMENT '与 create_time 反相关',
PRIMARY KEY (order_no, new_status) NOT ENFORCED
) with (
'merge-engine' = 'Deduplicate',
'sequence.field' = 'version'
);`
2. starting a flink job (consume kafka and write into paimon)
3. waiting some time, stop flink job with a savepoint
4. recover flink job with savepoint, flink job throw exception
5. restart flink job without savepoint, but same exception is occur
### What doesn't meet your expectations?
Even though I restarted the flink task stateless to write into paimon, the same error occurred:
FileNotFoundException
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Streaming writes to a Deduplicate table, flink job can't recover for FileNotFoundException [incubator-paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on issue #2273:
URL: https://github.com/apache/incubator-paimon/issues/2273#issuecomment-1813647356
It seems like you have recovered your job from an older savepoint.
If you are just using stop with savepoint, and recover your job, it is impossible to have this problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Streaming writes to a Deduplicate table, flink job can't recover for FileNotFoundException [incubator-paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on issue #2273:
URL: https://github.com/apache/incubator-paimon/issues/2273#issuecomment-1813647852
But I will add check to avoid this situation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Streaming writes to a Deduplicate table, flink job can't recover for FileNotFoundException [incubator-paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi closed issue #2273: [Bug] Streaming writes to a Deduplicate table, flink job can't recover for FileNotFoundException
URL: https://github.com/apache/incubator-paimon/issues/2273
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org