You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Caizhi Weng (Jira)" <ji...@apache.org> on 2022/03/02 05:43:00 UTC
[jira] [Created] (FLINK-26442) Allow ManifestFile to write a list of ManifestFileMeta into multiple files if too many.
Caizhi Weng created FLINK-26442:
-----------------------------------
Summary: Allow ManifestFile to write a list of ManifestFileMeta into multiple files if too many.
Key: FLINK-26442
URL: https://issues.apache.org/jira/browse/FLINK-26442
Project: Flink
Issue Type: Sub-task
Components: Table Store
Affects Versions: 0.1.0
Reporter: Caizhi Weng
Currently {{ManifestFile#write}} will only produce a single file no matter how large the given {{ManifestFileMeta}} list is. However batch jobs are likely to produce a lot of changes hence the list might be very long. We need to allow {{ManifestFile#write}} to produce multiple files for effective merging and filtering.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)