You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Haibo Chen (JIRA)" <ji...@apache.org> on 2018/03/12 23:19:00 UTC
[jira] [Commented] (YARN-5627) [Atsv2] Support streaming reader API
to fetch entities
[ https://issues.apache.org/jira/browse/YARN-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396234#comment-16396234 ]
Haibo Chen commented on YARN-5627:
----------------------------------
This has come up in next-gen JHS with ATSv2 exploration that I was doing. It is necessary to support extreme large jobs (500,000 tasks and above).
> [Atsv2] Support streaming reader API to fetch entities
> ------------------------------------------------------
>
> Key: YARN-5627
> URL: https://issues.apache.org/jira/browse/YARN-5627
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelinereader
> Reporter: Rohith Sharma K S
> Assignee: Rohith Sharma K S
> Priority: Major
>
> There is no limit for size of TimelineEntitie object. It can be varied from KB's to MB. While reading entities list, it would be an potential issue that TimeLineReder would go into OOM situation based on the entity size and limit.
> Proposal is to support an streaming API to read entities list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org