You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2024/12/27 06:05:39 UTC
[PR] [SPARK-50310][CONNECT][PYTHON][FOLLOW-UP] Delay is_debugging_enabled call after modules are initialized [spark]
HyukjinKwon opened a new pull request, #49311:
URL: https://github.com/apache/spark/pull/49311
### What changes were proposed in this pull request?
This PR is a retry of https://github.com/apache/spark/pull/49054 that avoids hacky monkey patch.
### Why are the changes needed?
- This disables DataFrameQueryContext for `pyspark.sql.functions` too
- It avoids circular import in pyspark-connect package.
### Does this PR introduce _any_ user-facing change?
Yes, after this followup, `spark.python.sql.dataFrameDebugging.enabled` also works with `pyspark.sql.functions.*`.
### How was this patch tested?
Manually ran profilers:
```python
import cProfile
from pyspark.sql.functions import col
def foo():
for _ in range(1000):
col("id")
cProfile.run('foo()', sort='tottime')
```
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50310][CONNECT][PYTHON][FOLLOW-UP] Delay is_debugging_enabled call after modules are initialized [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #49311:
URL: https://github.com/apache/spark/pull/49311#issuecomment-2563357687
gonna merge this because it's gonna recover the build after https://github.com/apache/spark/commit/1a1fcd61322b3cf9713e987a16882d7ddf666d2e revert.
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50310][CONNECT][PYTHON][FOLLOW-UP] Delay is_debugging_enabled call after modules are initialized [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #49311:
URL: https://github.com/apache/spark/pull/49311#issuecomment-2563356516
cc @itholic @zhengruifeng
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50310][CONNECT][PYTHON][FOLLOW-UP] Delay is_debugging_enabled call after modules are initialized [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #49311: [SPARK-50310][CONNECT][PYTHON][FOLLOW-UP] Delay is_debugging_enabled call after modules are initialized
URL: https://github.com/apache/spark/pull/49311
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org