[jira] [Created] (FLINK-10762) Do not try to request partitions in SingleInputGate per getRecord

["Nico Kruber (JIRA)" <ji...@apache.org>]

Nico Kruber created FLINK-10762:
-----------------------------------

             Summary: Do not try to request partitions in SingleInputGate per getRecord
                 Key: FLINK-10762
                 URL: https://issues.apache.org/jira/browse/FLINK-10762
             Project: Flink
          Issue Type: Improvement
          Components: Network
    Affects Versions: 1.7.0
            Reporter: Nico Kruber
             Fix For: 1.8.0


Every call to {{SingleInputGate#getNextBufferOrEvent()}} will also try to request partitions via {{SingleInputGate#requestPartitions()}}. However, in the current design, this must be under the {{requestLock}} which is costly and unnecessary.

I propose a redesign to change that and make the {{SingleInputGate#requestPartitions()}} a one-time only thing that should be called *before* going into any loop asking for new records. In that case, we could keep the lock and ignore and performance-related improvements.

Alternatively, if we had a separate {{unrequestedPartitions}} list aside to {{inputChannels}} and only go through these when executing {{SingleInputGate#requestPartitions()}}, we would change {{updateInputChannel}} to always request the newly created partition and will not have the concurrency issue between these two methods anymore. This {{unrequestedPartitions}} list would basically also make {{requestedPartitionsFlag}} obsolete if we, for example, set it to {{Optional.empty}} after finishing the requests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)