You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@apex.apache.org by "Yogi Devendra (JIRA)" <ji...@apache.org> on 2016/09/14 10:06:20 UTC

[jira] [Created] (APEXMALHAR-2237) Dynamic partitioning support for FSInputModule

Yogi Devendra created APEXMALHAR-2237:
-----------------------------------------

             Summary: Dynamic partitioning support for FSInputModule
                 Key: APEXMALHAR-2237
                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2237
             Project: Apache Apex Malhar
          Issue Type: Improvement
            Reporter: Yogi Devendra
            Assignee: Yogi Devendra


Currently, FSInputModule exposes readersCount property which is used for static partitioning of BlockReader.

To support dynamic partitioning, following additional properties will be exposed.
1. minReaders = minimum number of readers for dynamic partitioning.
2. maxReaders = maximum number of readers for dynamic partitioning.
3. repartitionCheckInterval = time in seconds for checking if repartition is required.

minReaders, maxReaders will set respective property on BlockReader.
repartitionCheckInterval will set intervalMillis property on BlockReader after converting from seconds to milliseconds. 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)