You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "LuciferYang (via GitHub)" <gi...@apache.org> on 2026/10/16 13:34:55 UTC
[PR] [flink] Restore rewrite_file_index jobs from checkpoints [paimon]
LuciferYang opened a new pull request, #10615:
URL: https://github.com/apache/paimon/pull/10615
### Purpose
close #10614
`RewriteFileIndexSource.CheckpointSerde.serialize` wrote the remaining splits into a `ByteArrayOutputStream` and then returned `new byte[0]`. Restoring the enumerator from a checkpoint or savepoint read an empty array, so a `rewrite_file_index` job that was stopped with a savepoint, or that went through a global or JobManager failover after its first checkpoint, could not be restored. It now returns the written bytes.
### Tests
`RewriteFileIndexSourceITCase#testRestoreFromSavepoint`: runs the topology of `rewrite_file_index` with checkpointing, stops it with a savepoint after its first commit while some files are still unindexed, and restores it. Every data file ends up with a file index and the row count is unchanged. On master the restored job does not finish.
### API and Format
no
### Documentation
no
Generated-by: Claude Opus
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [flink] Restore rewrite_file_index jobs from checkpoints [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #10615:
URL: https://github.com/apache/paimon/pull/10615#issuecomment-6247869270
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [flink] Restore rewrite_file_index jobs from checkpoints [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #10615:
URL: https://github.com/apache/paimon/pull/10615
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org