You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "TezQA (JIRA)" <ji...@apache.org> on 2017/06/22 13:14:00 UTC
[jira] [Commented] (TEZ-3769) Unordered: Fix wrong stats being sent
out in the last event, when final merge is disabled
[ https://issues.apache.org/jira/browse/TEZ-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059318#comment-16059318 ]
TezQA commented on TEZ-3769:
----------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12874074/TEZ-3769.1.patch
against master revision a925c83.
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files.
{color:red}-1 javac{color}. The applied patch generated 23 javac compiler warnings (more than the master's current 21 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/2535//testReport/
Javac warnings: https://builds.apache.org/job/PreCommit-TEZ-Build/2535//artifact/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2535//console
This message is automatically generated.
> Unordered: Fix wrong stats being sent out in the last event, when final merge is disabled
> -----------------------------------------------------------------------------------------
>
> Key: TEZ-3769
> URL: https://issues.apache.org/jira/browse/TEZ-3769
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Attachments: TEZ-3769.1.patch
>
>
> When final merge is disabled (without pipelining), wrong stats was sent out in the last event.
> It was based on {{numRecordsPerPartition}} which contains the overall partition data. It should be ideally be based on the spill result and its buffers.
> Also, {{finalSpill}} was unncessarily sending events when no data was present (i.e, when currentBuffer didn't have any data). This can be optimized to reduce the number of events being sent across.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)