[jira] [Issue Comment Deleted] (TEZ-2834) tez app hangs at large scale (~30TB)

["Bikas Saha (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bikas Saha updated TEZ-2834:
----------------------------
    Comment: was deleted

(was: {color:red}-1 overall{color}.  Here are the results of testing the latest attachment
  http://issues.apache.org/jira/secure/attachment/12761258/TEZ-2834.2.patch
  against master revision 7ed7025.

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 3 new or modified test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version 3.0.1) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:red}-1 core tests{color}.  The patch failed these unit tests in :
                   org.apache.tez.test.TestFaultTolerance

Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1158//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1158//console

This message is automatically generated.)

> tez app hangs at large scale (~30TB)
> ------------------------------------
>
>                 Key: TEZ-2834
>                 URL: https://issues.apache.org/jira/browse/TEZ-2834
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.7.1
>            Reporter: Rajesh Balamohan
>            Assignee: Bikas Saha
>         Attachments: DAG_view.png, TEZ-2834.1.patch, TEZ-2834.2.patch, application_1442254312093_0095.1.log.gz, application_1442254312093_0095.2.log.gz, hive_view.png
>
>
> Will attach the DAG.
> Repro for reference: TPC-DS q_70 @ 30 TB scale.
> "Map 7" completes in 2 waves. Output is very tiny, so reducer 8 gets launched slightly late.  But before "Reducer 9" can get scheduled, slots are taken up by "Map 1", which is not preempted for running "Reducer 9".
> This is with 0.7.1 codebase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)