You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/06/08 13:05:55 UTC
[GitHub] [iceberg] chenwyi2 opened a new issue, #4996: rewritedatafile: Cannot commit, found new position delete for replaced data
chenwyi2 opened a new issue, #4996:
URL: https://github.com/apache/iceberg/issues/4996
i use spark to run rewritedatafile， and i have a flink job write this table every 5 min, but if the rewritefatafile spend more than 5 min, it will commit error with "ValidationException: Cannot commit, found new position delete for replaced data file: GenericDataFile{content=data, file_path=qbfs://online01/warehouse/prod_censor_datalake.db/document/dataxxx", so there is a good solution? so now i must keep rewritedatafile in 5 min, otherwise it will commit failed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] humengyu2012 commented on issue #4996: rewritedatafile: Cannot commit, found new position delete for replaced data
Posted by GitBox <gi...@apache.org>.
humengyu2012 commented on issue #4996:
URL: https://github.com/apache/iceberg/issues/4996#issuecomment-1331235100
My iceberg version is 1.0.0, but the same problem occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] humengyu2012 commented on issue #4996: rewritedatafile: Cannot commit, found new position delete for replaced data
Posted by "humengyu2012 (via GitHub)" <gi...@apache.org>.
humengyu2012 commented on issue #4996:
URL: https://github.com/apache/iceberg/issues/4996#issuecomment-1429458376
Is there any recent progress on this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jingli430 commented on issue #4996: rewritedatafile: Cannot commit, found new position delete for replaced data
Posted by GitBox <gi...@apache.org>.
jingli430 commented on issue #4996:
URL: https://github.com/apache/iceberg/issues/4996#issuecomment-1162675524
I am seeing the same issue. I am ok If there is new position-delete found and then we cannot commit here, but could we have a strategy to auto-retry?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] chenwyi2 commented on issue #4996: rewritedatafile: Cannot commit, found new position delete for replaced data
Posted by GitBox <gi...@apache.org>.
chenwyi2 commented on issue #4996:
URL: https://github.com/apache/iceberg/issues/4996#issuecomment-1149907182
and i already use using use-starting-sequence-number variable, iceberg version is 0.13.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] chenwyi2 commented on issue #4996: rewritedatafile: Cannot commit, found new position delete for replaced data
Posted by GitBox <gi...@apache.org>.
chenwyi2 commented on issue #4996:
URL: https://github.com/apache/iceberg/issues/4996#issuecomment-1254690858
we use cdc mode, the data is always changing, even if we use strategy to auto-retry, rewritedatafile will always be failed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] anistal commented on issue #4996: rewritedatafile: Cannot commit, found new position delete for replaced data
Posted by "anistal (via GitHub)" <gi...@apache.org>.
anistal commented on issue #4996:
URL: https://github.com/apache/iceberg/issues/4996#issuecomment-1481867599
Same here, it is a big issue because we can not ingest and optimize at the same time. Is there any idea or workaround to solve it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org