You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by "Bhupesh Chawda (JIRA)" <ji...@apache.org> on 2017/03/17 08:52:41 UTC
[jira] [Updated] (APEXMALHAR-2449) Support for batch demarcation in
input/output operators
[ https://issues.apache.org/jira/browse/APEXMALHAR-2449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhupesh Chawda updated APEXMALHAR-2449:
---------------------------------------
Description:
In order to support the demarcation of multiple batches in an application, we need to adapt the I/O operators to be able to send and receive control tuples. These control tuples would serve multiple purposes:
1. They can be used as watermarks thereby allowing stateful windowed processing
2. They can also serve as batch demarcation tuples which allows us to separate the state for these batches in the intermediate as well as the output operators.
Discussion thread here: https://lists.apache.org/thread.html/b4b27a59ecfe506260f694f66d520ba1e906ca5b2f360b07e173039b@%3Cdev.apex.apache.org%3E
This would be worked on once apex-malhar depends on the release of apex-core which has control tuple support.
was:
In order to support the demarcation of multiple batches in an application, we need to adapt the I/O operators to be able to send and receive control tuples. These control tuples would serve multiple purposes:
1. They can be used as watermarks thereby allowing stateful windowed processing
2. They can also serve as batch demarcation tuples which allows us to separate the state for these batches in the intermediate as well as the output operators.
Discussion thread here: https://lists.apache.org/thread.html/b4b27a59ecfe506260f694f66d520ba1e906ca5b2f360b07e173039b@%3Cdev.apex.apache.org%3E
> Support for batch demarcation in input/output operators
> -------------------------------------------------------
>
> Key: APEXMALHAR-2449
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2449
> Project: Apache Apex Malhar
> Issue Type: New Feature
> Reporter: Bhupesh Chawda
> Assignee: Bhupesh Chawda
>
> In order to support the demarcation of multiple batches in an application, we need to adapt the I/O operators to be able to send and receive control tuples. These control tuples would serve multiple purposes:
> 1. They can be used as watermarks thereby allowing stateful windowed processing
> 2. They can also serve as batch demarcation tuples which allows us to separate the state for these batches in the intermediate as well as the output operators.
> Discussion thread here: https://lists.apache.org/thread.html/b4b27a59ecfe506260f694f66d520ba1e906ca5b2f360b07e173039b@%3Cdev.apex.apache.org%3E
> This would be worked on once apex-malhar depends on the release of apex-core which has control tuple support.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)