[GitHub] [samza] mynameborat commented on a change in pull request #1459: SAMZA-2618: Add per-container metric for memory-utilization

[GitBox <gi...@apache.org>]

mynameborat commented on a change in pull request #1459:
URL: https://github.com/apache/samza/pull/1459#discussion_r569032738



##########
File path: samza-core/src/main/scala/org/apache/samza/container/SamzaContainerMetrics.scala
##########
@@ -47,6 +47,7 @@ class SamzaContainerMetrics(
   val diskQuotaBytes = newGauge("disk-quota-bytes", Long.MaxValue)
   val executorWorkFactor = newGauge("executor-work-factor", 1.0)
   val physicalMemoryMb = newGauge[Double]("physical-memory-mb", 0.0F)
+  val containerMemoryUtilization = newGauge("container-memory-utilization", 0.0F)

Review comment:
       1. Can we be more specific and call this `physical-memory-utilization`?
   2. Can you update the [monitoring documentation](https://github.com/apache/samza/blob/master/docs/learn/documentation/versioned/operations/monitoring.md) as well 
   3. Do we need to explicitly pass the metric value type here? I see we have inconsistencies above where some pass the type `Double` and some don't. Will be good to maintain consistency one way or the other.
   

##########
File path: samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala
##########
@@ -617,13 +617,18 @@ object SamzaContainer extends Logging {
       taskConfig,
       clock)
 
+    val containerMemoryMb : Int = new ClusterManagerConfig(config).getContainerMemoryMb
+
     val memoryStatisticsMonitor : SystemStatisticsMonitor = new StatisticsMonitorImpl()
     memoryStatisticsMonitor.registerListener(new SystemStatisticsMonitor.Listener {
       override def onUpdate(sample: SystemMemoryStatistics): Unit = {
         val physicalMemoryBytes : Long = sample.getPhysicalMemoryBytes
         val physicalMemoryMb : Double = physicalMemoryBytes / (1024.0 * 1024.0)
+        val memoryUtilization : Float = physicalMemoryMb.toFloat / containerMemoryMb
         logger.debug("Container physical memory utilization (mb): " + physicalMemoryMb)
+        logger.debug("Container memory utilization: " + memoryUtilization)

Review comment:
       nit: be explicit about physical memory utilization




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org