You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/04/25 07:45:44 UTC
[PR] [SPARK-51898][PYTHON][CONNECT][TESTS][FOLLOW-UP] Fix inconsistent exception [spark]
zhengruifeng opened a new pull request, #50708:
URL: https://github.com/apache/spark/pull/50708
### What changes were proposed in this pull request?
`toJSON` and `rdd` throws `PySparkNotImplementedError` in connect model, but `PySparkAttributeError` in connect-only model
### Why are the changes needed?
to fix https://github.com/apache/spark/actions/runs/14649632571/job/41112060443
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
will be tested in daily builder
### Was this patch authored or co-authored using generative AI tooling?
no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-51898][PYTHON][CONNECT][TESTS][FOLLOW-UP] Fix inconsistent exception [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #50708:
URL: https://github.com/apache/spark/pull/50708#issuecomment-2833246737
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-51898][PYTHON][CONNECT][TESTS][FOLLOW-UP] Fix inconsistent exception [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #50708: [SPARK-51898][PYTHON][CONNECT][TESTS][FOLLOW-UP] Fix inconsistent exception
URL: https://github.com/apache/spark/pull/50708
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-51898][PYTHON][CONNECT][TESTS][FOLLOW-UP] Fix inconsistent exception [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #50708:
URL: https://github.com/apache/spark/pull/50708#discussion_r2059904698
##########
python/pyspark/sql/connect/dataframe.py:
##########
@@ -2190,20 +2190,18 @@ def localCheckpoint(
assert isinstance(checkpointed._plan, plan.CachedRemoteRelation)
return checkpointed
- if not is_remote_only():
Review Comment:
This is actually intended, e.g., for autocompletetion
##########
python/pyspark/sql/connect/dataframe.py:
##########
@@ -2190,20 +2190,18 @@ def localCheckpoint(
assert isinstance(checkpointed._plan, plan.CachedRemoteRelation)
return checkpointed
- if not is_remote_only():
Review Comment:
There are other cases in session.py as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-51898][PYTHON][CONNECT][TESTS][FOLLOW-UP] Fix inconsistent exception [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #50708:
URL: https://github.com/apache/spark/pull/50708#discussion_r2059968399
##########
python/pyspark/sql/connect/dataframe.py:
##########
@@ -2190,20 +2190,18 @@ def localCheckpoint(
assert isinstance(checkpointed._plan, plan.CachedRemoteRelation)
return checkpointed
- if not is_remote_only():
Review Comment:
so we should just skip the test? or remove here and always throw PySparkAttributeError?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org