You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Xuefu Zhang (JIRA)" <ji...@apache.org> on 2016/05/17 04:35:13 UTC
[jira] [Resolved] (PIG-4876) OutputConsumeIterator can't handle the
last buffered tuples for some Operators
[ https://issues.apache.org/jira/browse/PIG-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xuefu Zhang resolved PIG-4876.
------------------------------
Resolution: Fixed
Committed to Spark branch. Thanks, Xianda.
> OutputConsumeIterator can't handle the last buffered tuples for some Operators
> ------------------------------------------------------------------------------
>
> Key: PIG-4876
> URL: https://issues.apache.org/jira/browse/PIG-4876
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: Xianda Ke
> Assignee: Xianda Ke
> Fix For: spark-branch
>
> Attachments: PIG-4876.patch
>
>
> Some Operators, such as MergeCogroup, Stream, CollectedGroup etc buffer some input records to constitute the result tuples. The last result tuples are buffered in the operator. These Operators need a flag to indicate the end of input, so that they can flush and constitute their last tuples.
> Currently, the flag 'parentPlan.endOfAllInput' is targeted for flushing the buffered tuples in MR mode. But it does not work with OutputConsumeIterator in Spark mode.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)