You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/11/24 20:05:54 UTC

[GitHub] [spark] grundprinzip opened a new pull request, #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession

grundprinzip opened a new pull request, #38792:
URL: https://github.com/apache/spark/pull/38792

   ### What changes were proposed in this pull request?
   For better source compatibility, this PR changes the type name of RemoteSparkSession to SparkSession and follows the same builder pattern.
   
   ### Why are the changes needed?
   Compatibility.
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   ### How was this patch tested?
   UT


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] grundprinzip commented on pull request #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession

Posted by GitBox <gi...@apache.org>.
grundprinzip commented on PR #38792:
URL: https://github.com/apache/spark/pull/38792#issuecomment-1327211844

   I will split the session and client back out again to make it a bit easier to reason about.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] grundprinzip commented on pull request #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession

Posted by GitBox <gi...@apache.org>.
grundprinzip commented on PR #38792:
URL: https://github.com/apache/spark/pull/38792#issuecomment-1326809906

   R: @HyukjinKwon @cloud-fan @zhengruifeng 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] amaliujia commented on pull request #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession

Posted by GitBox <gi...@apache.org>.
amaliujia commented on PR #38792:
URL: https://github.com/apache/spark/pull/38792#issuecomment-1326905298

   1. It is ok for now. But we need to have a clear documentation or throws errors for the Connect's SparkSession, especially for those bits that does not work as PySpark SparkSession. It is because now the API looks compatible, then users will easy to assume they can do what they usually do with PySpark SparkSession, which is certainly causing confusions.
   
   2. I still think we should keep `client.py`. For the new `session.py`, there are compatible SparSession API. For dealing with gRPC and proto, we keep those in `client.py`. These two logically can be separated. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] hvanhovell closed pull request #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession

Posted by GitBox <gi...@apache.org>.
hvanhovell closed pull request #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession
URL: https://github.com/apache/spark/pull/38792


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] grundprinzip commented on pull request #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession

Posted by GitBox <gi...@apache.org>.
grundprinzip commented on PR #38792:
URL: https://github.com/apache/spark/pull/38792#issuecomment-1326841266

   > Per design, Is the RemoteSparkSession as powerful as the SparkSession? Is there a limitation?
   
   "as powerful" is a very strong statement. I don't think we have fully defined exactly what is going to be supported. Right now, the goal is to reflect the simplicity of the basics and try to be opinionated when it comes to configurations. The reason the session allows configuration values right now is purely for code compatibility purposes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] grundprinzip commented on pull request #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession

Posted by GitBox <gi...@apache.org>.
grundprinzip commented on PR #38792:
URL: https://github.com/apache/spark/pull/38792#issuecomment-1327885826

   @cloud-fan @hvanhovell @HyukjinKwon any chance you can merge this so that we can rebase the other changes? Thank you!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] amaliujia commented on pull request #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession

Posted by GitBox <gi...@apache.org>.
amaliujia commented on PR #38792:
URL: https://github.com/apache/spark/pull/38792#issuecomment-1327733869

   LGTM. Thanks for update the code structure! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] hvanhovell commented on pull request #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession

Posted by GitBox <gi...@apache.org>.
hvanhovell commented on PR #38792:
URL: https://github.com/apache/spark/pull/38792#issuecomment-1327911326

   merging


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on PR #38792:
URL: https://github.com/apache/spark/pull/38792#issuecomment-1327052861

   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] amaliujia commented on pull request #38792: [SPARK-41255] [CONNECT] Rename RemoteSparkSession

Posted by GitBox <gi...@apache.org>.
amaliujia commented on PR #38792:
URL: https://github.com/apache/spark/pull/38792#issuecomment-1326829478

   Per design, Is the RemoteSparkSession as powerful as the SparkSession (e.g. decide to launch cluster, etc.)?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org