You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Matthias (Jira)" <ji...@apache.org> on 2020/10/09 16:14:00 UTC
[jira] [Commented] (FLINK-14406) Add metric for managed memory
[ https://issues.apache.org/jira/browse/FLINK-14406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211127#comment-17211127 ]
Matthias commented on FLINK-14406:
----------------------------------
FYI: The {{managedMemory}} didn't end up in the {{TaskManagerDetailsHandler}} after [~chesnay] raised the idea of using the metrics REST API instead. I created FLINK-19559 to cleanup the {{TaskManagerDetailsHandler}} REST API accordingly.
> Add metric for managed memory
> -----------------------------
>
> Key: FLINK-14406
> URL: https://issues.apache.org/jira/browse/FLINK-14406
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics, Runtime / Task
> Reporter: lining
> Assignee: Matthias
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If a user wants to get memory used in time, as there's no manage memory's metrics, it couldn't get it.
> *Propose*
> * add default memory type in MemoryManager
>
> {code:java}
> public static final MemoryType DEFAULT_MEMORY_TYPE = MemoryType.OFF_HEAP;
> {code}
> * add getManagedMemoryTotal in TaskExecutor:
>
> {code:java}
> public long getManagedMemoryTotal() {
> return this.taskSlotTable.getAllocatedSlots().stream().mapToLong(
> slot -> slot.getMemoryManager().getMemorySizeByType(MemoryManager.DEFAULT_MEMORY_TYPE)
> ).sum();
> }
> {code}
>
> * add getManagedMemoryUsed in TaskExecutor:
>
> {code:java}
> public long getManagedMemoryUsed() {
> return this.taskSlotTable.getAllocatedSlots().stream().mapToLong(
> slot -> slot.getMemoryManager().getMemorySizeByType(MemoryManager.DEFAULT_MEMORY_TYPE) - slot.getMemoryManager().availableMemory(MemoryManager.DEFAULT_MEMORY_TYPE)
> ).sum();
> }
> {code}
>
> * add instantiateMemoryManagerMetrics in MetricUtils
>
> {code:java}
> public static void instantiateMemoryManagerMetrics(MetricGroup statusMetricGroup, TaskExecutor taskExecutor) {
> checkNotNull(statusMetricGroup);
> MetricGroup memoryManagerGroup = statusMetricGroup.addGroup("Managed").addGroup("Memory");
> memoryManagerGroup.<Long, Gauge<Long>>gauge("TotalCapacity", taskExecutor::getManagedMemoryTotal);
> memoryManagerGroup.<Long, Gauge<Long>>gauge("MemoryUsed", taskExecutor::getManagedMemoryUsed);
> }
> {code}
> * register it in TaskManagerRunner#startTaskManager
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)