You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Jonathan Eagles (JIRA)" <ji...@apache.org> on 2019/07/23 19:43:00 UTC

[jira] [Commented] (TEZ-4079) Limit the size of broadcast data

    [ https://issues.apache.org/jira/browse/TEZ-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891322#comment-16891322 ] 

Jonathan Eagles commented on TEZ-4079:
--------------------------------------

Perhaps logic could be added to BroadcastEdgeManager to fail if a max threshold is breached. Broadcast size can only be know during runtime currently.

> Limit the size of broadcast data
> --------------------------------
>
>                 Key: TEZ-4079
>                 URL: https://issues.apache.org/jira/browse/TEZ-4079
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Jonathan Eagles
>            Priority: Major
>
> Since broadcast data is fetched all from a single node currently. Large broadcast data can DDOS shuffle handler with sufficiently high number of downstream tasks.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)