You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/11/28 10:18:56 UTC
[GitHub] [iceberg] humengyu2012 opened a new issue, #6288: Can not recovery flink job by savepoint
humengyu2012 opened a new issue, #6288:
URL: https://github.com/apache/iceberg/issues/6288
### Apache Iceberg version
0.13.1
### Query engine
Flink
### Please describe the bug 🐞
1. I have a flink job, flink cdc->iceberg;
2. stop flink job by savepoint;
3. compact file by the following code:
```java
LOG.info("Start delete orphan for {}.{}", databaseName, tableName);
// Delete orphan files
table = catalog.loadTable(tableIdentifier);
SparkActions
.get()
.deleteOrphanFiles(table)
.execute();
LOG.info("Start rewrite manifests for {}.{}", databaseName, tableName);
// rewrite Manifests
table = catalog.loadTable(tableIdentifier);
SparkActions
.get()
.rewriteManifests(table)
.rewriteIf(file -> file.length() < 10 * 1024 * 1024)
.execute();
LOG.info("Start compact data for {}.{}", databaseName, tableName);
// compact data files, 100M
table = catalog.loadTable(tableIdentifier);
SparkActions
.get()
.rewriteDataFiles(table)
.option(TARGET_FILE_SIZE_BYTES, Long.toString(100 * 1024 * 1024))
.option(MAX_CONCURRENT_FILE_GROUP_REWRITES, Integer.toString(10))
.option(PARTIAL_PROGRESS_ENABLED, "true")
.execute();
```
4. recovery flink job by savepoint, bu exception is thrown:
<img width="1414" alt="企业微信截图_bd91c630-3bd5-4d55-80ee-ec671e40cfa7" src="https://user-images.githubusercontent.com/45056332/204252906-01b1ec2c-833d-4ceb-8226-8dd345d92fc7.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] humengyu2012 closed issue #6288: Can not recovery flink job by savepoint
Posted by GitBox <gi...@apache.org>.
humengyu2012 closed issue #6288: Can not recovery flink job by savepoint
URL: https://github.com/apache/iceberg/issues/6288
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org