[GitHub] LucaCanali opened a new pull request #23838: [SPARK-26928][CORE] Add driver CPU Time to the metrics system

[GitBox <gi...@apache.org>]

LucaCanali opened a new pull request #23838: [SPARK-26928][CORE] Add driver CPU Time to the metrics system
URL: https://github.com/apache/spark/pull/23838
 
 
   ## What changes were proposed in this pull request?
   
   This proposes to add instrumentation for the driver's JVM CPU time via the Spark Dropwizard/Codahale metrics system. It follows directly from previous work SPARK-25228 and shares similar motivations: it is intended as an improvement to be used for Spark performance dashboards and monitoring tools/instrumentation.
   
   Implementation details: this PR takes the code introduced in SPARK-25228 and moves it to a new separate Source JVMCPUSource, which is then used to register the jvmCpuTime gauge metric for both executor and driver.
   The registration of the jvmCpuTime metric for the driver is conditional, a new configuration parameter `spark.metrics.cpu.time.driver.enabled` (proposed default: false) is introduced for this purpose.
   
   ## How was this patch tested?
   
   Manually tested, using local mode and using YARN.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org