You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2024/01/24 02:29:00 UTC
[PR] [SPARK-46530][PYTHON][SQL][FOLLOW-UP] Avoid checking Py4J and PySpark library when initial lookup of Python Data Sources [spark]
HyukjinKwon opened a new pull request, #44858:
URL: https://github.com/apache/spark/pull/44858
### What changes were proposed in this pull request?
This PR partially reverts https://github.com/apache/spark/commit/d6334a3ba87c39fff6ace04e43e760d86674551e (with https://github.com/apache/spark/commit/b303eced7f8639887278db34e0080ffa0c19bd0c) by removing the check of Py4J and PySpark libraries when initial lookup of Python Data Sources.
### Why are the changes needed?
We actually guard the case by try-catch, and system-installed Py4J could be used instead. In addition, PySpark source location might vary as well.
### Does this PR introduce _any_ user-facing change?
Virtually, no. The main change has not been released yet. It will relax the condition of loading initial Python Data Sources.
### How was this patch tested?
Manually tested.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46530][PYTHON][SQL][FOLLOW-UP] Avoid checking Py4J and PySpark library when initial lookup of Python Data Sources [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #44858: [SPARK-46530][PYTHON][SQL][FOLLOW-UP] Avoid checking Py4J and PySpark library when initial lookup of Python Data Sources
URL: https://github.com/apache/spark/pull/44858
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46530][PYTHON][SQL][FOLLOW-UP] Avoid checking Py4J and PySpark library when initial lookup of Python Data Sources [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #44858:
URL: https://github.com/apache/spark/pull/44858#issuecomment-1907312443
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org