[jira] [Commented] (TEZ-668) Allow a Processor to trigger an Input as to when to fetch/prep data.

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898306#comment-13898306 ] 

Siddharth Seth commented on TEZ-668:
------------------------------------

Splitting this into two - one to just separate the intialize and start functionality, the other to expose an interface for users to control this.

The API in the current patch is kindof ugly. [~hagleitn] had a suggestion on exposing this via the DAG.Edge API itself - something like IOs AutoStart by default, but can be controlled to never auto-start / auto start once per JVM via the Edge itself.

> Allow a Processor to trigger an Input as to when to fetch/prep data.
> --------------------------------------------------------------------
>
>                 Key: TEZ-668
>                 URL: https://issues.apache.org/jira/browse/TEZ-668
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Hitesh Shah
>            Assignee: Siddharth Seth
>         Attachments: TEZ-668.1.wip.txt, TEZ-668.2.wip.txt
>
>
> In cases of container re-use or other scenarios, a processor may not need to process any data from a particular Input. Currently, an Input always fetches/preps the data as soon as its init() function is invoked. 
> A Processor should have a way to trigger the Input to fetch data. This obviously has some overheads of delaying the fetch. 
> The approach should be such that there is no additional burden on developers who write new processors when they need the normal flow of their inputs always fetching on init().



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)