You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Nishanth (Jira)" <ji...@apache.org> on 2025/11/06 17:01:00 UTC

[jira] [Created] (SPARK-54223) Add Task Context Information to PythonRunner Logs

Nishanth created SPARK-54223:
--------------------------------

             Summary: Add Task Context Information to PythonRunner Logs
                 Key: SPARK-54223
                 URL: https://issues.apache.org/jira/browse/SPARK-54223
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 4.0.1, 3.5.1
            Reporter: Nishanth


Currently, log messages in {{PythonRunner}} (used to execute Python UDFs) do not include Spark Task Context information such as Task ID or Partition ID.

When debugging UDF performance issues or hangs, it’s difficult to trace which Spark task or partition corresponds to the specific Python process that logged the message.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org