[jira] [Created] (FLINK-31008) [Flink][Table Store] The Split allocation of the same bucket in ContinuousFileSplitEnumerator may be out of order

["ming li (Jira)" <ji...@apache.org>]

ming li created FLINK-31008:
-------------------------------

             Summary: [Flink][Table Store] The Split allocation of the same bucket in ContinuousFileSplitEnumerator may be out of order
                 Key: FLINK-31008
                 URL: https://issues.apache.org/jira/browse/FLINK-31008
             Project: Flink
          Issue Type: Bug
          Components: Table Store
            Reporter: ming li


There are two places in {{ContinuousFileSplitEnumerator}} that add {{FileStoreSourceSplit}} to {{{}bucketSplits{}}}: {{addSplitsBack}} and {{{}processDiscoveredSplits{}}}. {{processDiscoveredSplits}} will continuously check for new splits and add them to the queue.  At this time, the order of the splits is in order.
{code:java}
private void addSplits(Collection<FileStoreSourceSplit> splits) {
    splits.forEach(this::addSplit);
}

private void addSplit(FileStoreSourceSplit split) {
    bucketSplits
            .computeIfAbsent(((DataSplit) split.split()).bucket(), i -> new LinkedList<>())
            .add(split);
}{code}
However, when the task failover, the splits that have been allocated before will be returned. At this time, these returned splits are also added to the end of the queue, which leads to disorder in the allocation of splits.

 

I think these returned splits should be added to the head of the queue to ensure the order of allocation.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)