You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "angerszhu (Jira)" <ji...@apache.org> on 2022/05/18 12:06:00 UTC
[jira] [Updated] (SPARK-39224) Ignore noisy. warning message in ProcfsMetricsGetter
[ https://issues.apache.org/jira/browse/SPARK-39224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
angerszhu updated SPARK-39224:
------------------------------
Summary: Ignore noisy. warning message in ProcfsMetricsGetter (was: Ignore noisy. warning message in ProcessMetricsGetter)
> Ignore noisy. warning message in ProcfsMetricsGetter
> ----------------------------------------------------
>
> Key: SPARK-39224
> URL: https://issues.apache.org/jira/browse/SPARK-39224
> Project: Spark
> Issue Type: Task
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: angerszhu
> Priority: Major
>
> There are many user complain about the noisy warning messages in ProcessMetricsGetter
> {code:java}
> 22/05/18 16:48:58 WARN ProcfsMetricsGetter: There was a problem with reading the stat file of the process.
> java.io.FileNotFoundException: /proc/50371/stat (No such file or directory)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.<init>(FileInputStream.java:138)
> at org.apache.spark.executor.ProcfsMetricsGetter.openReader$1(ProcfsMetricsGetter.scala:174)
> at org.apache.spark.executor.ProcfsMetricsGetter.$anonfun$addProcfsMetricsFromOneProcess$1(ProcfsMetricsGetter.scala:176)
> at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:2647)
> at org.apache.spark.executor.ProcfsMetricsGetter.addProcfsMetricsFromOneProcess(ProcfsMetricsGetter.scala:176)
> at org.apache.spark.executor.ProcfsMetricsGetter.$anonfun$computeAllMetrics$1(ProcfsMetricsGetter.scala:216)
> at scala.runtime.java8.JFunction1$mcVI$sp.apply(JFunction1$mcVI$sp.java:23)
> at scala.collection.immutable.Set$Set2.foreach(Set.scala:132)
> at org.apache.spark.executor.ProcfsMetricsGetter.computeAllMetrics(ProcfsMetricsGetter.scala:214)
> at org.apache.spark.metrics.ProcessTreeMetrics$.getMetricValues(ExecutorMetricType.scala:93)
> at org.apache.spark.executor.ExecutorMetrics$.$anonfun$getCurrentMetrics$1(ExecutorMetrics.scala:103)
> at org.apache.spark.executor.ExecutorMetrics$.$anonfun$getCurrentMetrics$1$adapted(ExecutorMetrics.scala:102)
> at scala.collection.Iterator.foreach(Iterator.scala:941)
> at scala.collection.Iterator.foreach$(Iterator.scala:941)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
> at scala.collection.IterableLike.foreach(IterableLike.scala:74)
> at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
> at org.apache.spark.executor.ExecutorMetrics$.getCurrentMetrics(ExecutorMetrics.scala:102)
> at org.apache.spark.SparkContext.reportHeartBeat(SparkContext.scala:2578)
> at org.apache.spark.SparkContext.$anonfun$new$30(SparkContext.scala:587)
> at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:2022)
> at org.apache.spark.Heartbeater$$anon$1.run(Heartbeater.scala:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org