You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2024/04/07 06:22:43 UTC
[PR] [SPARK-47753][PYTHON][TESTS] Make pyspark.testing compatible with pyspark-connect [spark]
HyukjinKwon opened a new pull request, #45916:
URL: https://github.com/apache/spark/pull/45916
### What changes were proposed in this pull request?
This PR proposes to make `pyspark.testing` compatible with `pyspark-connect` by using noop context manager `contextlib.nullcontext` instead of `QuietTest` which requires JVM access.
### Why are the changes needed?
In order for `pyspark-connect` to work without classic PySpark packages and dependencies. Also, the logs are hidden as it's written to the separate file so it is actually already quiet.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Yes, at https://github.com/apache/spark/pull/45870. Once CI is setup there, it will be tested there properly.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47753][PYTHON][CONNECT][TESTS] Make pyspark.testing compatible with pyspark-connect [spark]
Posted by "09306677806 (via GitHub)" <gi...@apache.org>.
09306677806 commented on PR #45916:
URL: https://github.com/apache/spark/pull/45916#issuecomment-2041407341
Is everything done correctly and entered into the teacher's account?
ShahrzadMahro
در تاریخ یکشنبه ۷ آوریل ۲۰۲۴، ۰۹:۵۴ Hyukjin Kwon ***@***.***>
نوشت:
> What changes were proposed in this pull request?
>
> This PR proposes to make pyspark.testing compatible with pyspark-connect
> by using noop context manager contextlib.nullcontext instead of QuietTest
> which requires JVM access.
> Why are the changes needed?
>
> In order for pyspark-connect to work without classic PySpark packages and
> dependencies. Also, the logs are hidden as it's written to the separate
> file so it is actually already quiet.
> Does this PR introduce *any* user-facing change?
>
> No.
> How was this patch tested?
>
> Yes, at #45870 <https://github.com/apache/spark/pull/45870>. Once CI is
> setup there, it will be tested there properly.
> Was this patch authored or co-authored using generative AI tooling?
>
> No.
> ------------------------------
> You can view, comment on, or merge this pull request online at:
>
> https://github.com/apache/spark/pull/45916
> Commit Summary
>
> - 41a0f59
> <https://github.com/apache/spark/pull/45916/commits/41a0f597d7a341470392d03462de6e8aa064d1af>
> Make pyspark.testing compatible with pyspark-connect
>
> File Changes
>
> (1 file <https://github.com/apache/spark/pull/45916/files>)
>
> - *M* python/pyspark/testing/connectutils.py
> <https://github.com/apache/spark/pull/45916/files#diff-a23e7cd23afd2d04a82b9e08fc86c68b74e9cc84e1e8bee23da18d4b64c8c4b5>
> (11)
>
> Patch Links:
>
> - https://github.com/apache/spark/pull/45916.patch
> - https://github.com/apache/spark/pull/45916.diff
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/45916>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ANEZZRAYDBEIIFV2TCNY6QTY4DRDVAVCNFSM6AAAAABF3B2LYSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGIZDSNJYG44DGNA>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47753][PYTHON][CONNECT][TESTS] Make pyspark.testing compatible with pyspark-connect [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #45916: [SPARK-47753][PYTHON][CONNECT][TESTS] Make pyspark.testing compatible with pyspark-connect
URL: https://github.com/apache/spark/pull/45916
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47753][PYTHON][CONNECT][TESTS] Make pyspark.testing compatible with pyspark-connect [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #45916:
URL: https://github.com/apache/spark/pull/45916#issuecomment-2041386906
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org