You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Yi Pan (Data Infrastructure) (JIRA)" <ji...@apache.org> on 2016/07/27 22:38:20 UTC
[jira] [Updated] (SAMZA-972) Add holistic memory monitoring for
SamzaContainer
[ https://issues.apache.org/jira/browse/SAMZA-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yi Pan (Data Infrastructure) updated SAMZA-972:
-----------------------------------------------
Attachment: SAMZA-972-1.patch
Rebased against latest master based on the RB
> Add holistic memory monitoring for SamzaContainer
> -------------------------------------------------
>
> Key: SAMZA-972
> URL: https://issues.apache.org/jira/browse/SAMZA-972
> Project: Samza
> Issue Type: Bug
> Reporter: Jagadish
> Assignee: Jagadish
> Fix For: 0.11.0
>
> Attachments: SAMZA-972-1.patch
>
>
> This feature introduces physical memory monitoring in SamzaContainer.
> Context:
> Often memory used by the SamzaContainer process includes
> A. JVM Heap memory: This is where all JVM variables live.
> B. Native memory: This memory lives out of the JVM heap and is not visible to the JVM. Examples include used by RocksDb, native libraries that user code depends on etc.
> User jobs could be killed by Yarn if their total memory (A+B) exceeds the configured maximum of yarn.container.memory.mb.
> Currently, while our existing metrics provide visibility into [A] via JMX, we don't have visibility into [B]. (as it's totally external to the JVM).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)