You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2021/11/29 05:39:32 UTC

[spark] branch master updated (db9a982 -> e91ef19)

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.


    from db9a982  [SPARK-37461][YARN] YARN-CLIENT mode client.appId is always null
     add e91ef19  [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

No new revisions were added by this update.

Summary of changes:
 dev/sparktestsupport/modules.py                    |  1 +
 python/docs/source/development/debugging.rst       | 56 ++++++++++++-
 python/pyspark/context.py                          | 10 ++-
 python/pyspark/context.pyi                         |  1 +
 python/pyspark/profiler.py                         | 45 +++++++++--
 python/pyspark/profiler.pyi                        | 17 +++-
 .../tests/test_udf_profiler.py}                    | 91 +++++++++++-----------
 python/pyspark/sql/udf.py                          | 32 ++++++--
 .../sql/catalyst/expressions/Expression.scala      | 11 +++
 .../spark/sql/catalyst/expressions/PythonUDF.scala | 20 ++++-
 .../catalyst/expressions/namedExpressions.scala    | 10 ---
 .../apache/spark/sql/catalyst/util/package.scala   |  1 +
 .../apache/spark/sql/IntegratedUDFTestUtils.scala  | 35 ++++++++-
 13 files changed, 251 insertions(+), 79 deletions(-)
 copy python/pyspark/{tests/test_profiler.py => sql/tests/test_udf_profiler.py} (52%)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org