[jira] [Comment Edited] (TEZ-1596) Secure Shuffle utils is extremely expensive for fast queries

["Rajesh Balamohan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182139#comment-14182139 ] 

Rajesh Balamohan edited comment on TEZ-1596 at 10/23/14 11:07 PM:
------------------------------------------------------------------

Adding the profiler snapshot of yesterday's run with latest codebase.  

[~sseth], it would be good to fix in 0.5.2 as it helps reducing the CPU usage.  Might have to slightly deviate from YARN codebase for this.

ShuffleHandler on YARN would have similar issue (but that can't be avoided at this time).


was (Author: rajesh.balamohan):
Adding the profiler snapshot of yesterday's run with latest codebase.  

@sseth, it would be good to fix in 0.5.2 as it helps reducing the CPU usage.  Might have to slightly deviate from YARN codebase for this.

ShuffleHandler on YARN would have similar issue (but that can't be avoided at this time).

> Secure Shuffle utils is extremely expensive for fast queries
> ------------------------------------------------------------
>
>                 Key: TEZ-1596
>                 URL: https://issues.apache.org/jira/browse/TEZ-1596
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.6.0
>            Reporter: Gopal V
>         Attachments: Shuffle_generateHash.png, TEZ-1596.hack.patch, shuffle-secure-drilldown.png, shuffle-secure.png
>
>
> Generating the hash for YARN's secure shuffle is more expensive than the actual HTTP call once keep-alive is turned on.
> !shuffle-secure.png!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)