You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Fabian Hueske (JIRA)" <ji...@apache.org> on 2014/10/09 15:31:34 UTC

[jira] [Created] (FLINK-1149) Operators cannot adapt DOP for NonParallelInputs

Fabian Hueske created FLINK-1149:
------------------------------------

             Summary: Operators cannot adapt DOP for NonParallelInputs
                 Key: FLINK-1149
                 URL: https://issues.apache.org/jira/browse/FLINK-1149
             Project: Flink
          Issue Type: Bug
          Components: Java API
    Affects Versions: 0.6.1-incubating, 0.7-incubating
            Reporter: Fabian Hueske
            Assignee: Fabian Hueske
            Priority: Minor


InputFormats that cannot be processed in parallel implement the NonParallelInput interface.
During optimization, the optimizer checks for this interface and sets the DOP of an operator to 1 if it is found. Other operators such as Mappers set their DOP during program construction to the DOP of their preceding task (if not specified otherwise). Since non-splittable data sources are only considered later by the optimizer, a Map operator will not have the same DOP as an preceding non-splittable data source.

The simple solution is to set the DOP of a non-splittable data source during program construction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)