[GitHub] [spark] vinodkc commented on a diff in pull request #41144: [SPARK-43470][CORE] Add OS, Java, Python version information to application log

["vinodkc (via GitHub)" <gi...@apache.org>]

vinodkc commented on code in PR #41144:
URL: https://github.com/apache/spark/pull/41144#discussion_r1217472619


##########
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala:
##########
@@ -106,6 +106,7 @@ private[spark] abstract class BasePythonRunner[IN, OUT](
   protected val envVars: java.util.Map[String, String] = funcs.head.funcs.head.envVars
   protected val pythonExec: String = funcs.head.funcs.head.pythonExec
   protected val pythonVer: String = funcs.head.funcs.head.pythonVer
+  logInfo(s"Python version info: $pythonExec($pythonVer)")

Review Comment:
   Updated code to print version and package details only once per executor
   Added a bit more details to log list of installed packages too. 
   A new property `spark.executor.python.worker.log.details` has been added to enable this Python info  logging 
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org