You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2023/03/21 07:49:28 UTC
[GitHub] [spark] HyukjinKwon opened a new pull request, #40505: [MINOR][DOCS] Remove SparkSession constructor invocation in the example
HyukjinKwon opened a new pull request, #40505:
URL: https://github.com/apache/spark/pull/40505
### What changes were proposed in this pull request?
This PR proposes to Remove SparkSession constructor invocation in the example.
### Why are the changes needed?
SparkSession's constructor is not meant to be exposed to the end users. This is also hidden in Scala side.
### Does this PR introduce _any_ user-facing change?
Yes, it removes the usage of SparkSession constructor in the user facing docs.
### How was this patch tested?
Linters should verify the changes in the CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bjornjorgensen commented on a diff in pull request #40505: [MINOR][DOCS] Remove SparkSession constructor invocation in the example
Posted by "bjornjorgensen (via GitHub)" <gi...@apache.org>.
bjornjorgensen commented on code in PR #40505:
URL: https://github.com/apache/spark/pull/40505#discussion_r1143270035
##########
python/pyspark/sql/session.py:
##########
@@ -179,10 +179,15 @@ class SparkSession(SparkConversionMixin):
... .getOrCreate()
... )
- Create a Spark session from a Spark context.
+ Create a Spark session with Spark Connect.
- >>> sc = spark.sparkContext
- >>> spark = SparkSession(sc)
+ >>> spark = (
+ ... SparkSession.builder
+ ... .remote("sc://localhost")
Review Comment:
add the port her?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] allisonwang-db commented on a diff in pull request #40505: [MINOR][DOCS] Remove SparkSession constructor invocation in the example
Posted by "allisonwang-db (via GitHub)" <gi...@apache.org>.
allisonwang-db commented on code in PR #40505:
URL: https://github.com/apache/spark/pull/40505#discussion_r1142993161
##########
python/pyspark/sql/session.py:
##########
@@ -178,11 +178,6 @@ class SparkSession(SparkConversionMixin):
... .config("spark.some.config.option", "some-value")
... .getOrCreate()
... )
Review Comment:
Should we add another example for creating a remote Spark session?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #40505: [MINOR][DOCS] Remove SparkSession constructor invocation in the example
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #40505:
URL: https://github.com/apache/spark/pull/40505#discussion_r1144108734
##########
python/pyspark/sql/session.py:
##########
@@ -179,10 +179,15 @@ class SparkSession(SparkConversionMixin):
... .getOrCreate()
... )
- Create a Spark session from a Spark context.
+ Create a Spark session with Spark Connect.
- >>> sc = spark.sparkContext
- >>> spark = SparkSession(sc)
+ >>> spark = (
+ ... SparkSession.builder
+ ... .remote("sc://localhost")
Review Comment:
That's fine without a port - it uses the default port 15002.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #40505: [MINOR][DOCS] Remove SparkSession constructor invocation in the example
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #40505: [MINOR][DOCS] Remove SparkSession constructor invocation in the example
URL: https://github.com/apache/spark/pull/40505
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #40505: [MINOR][DOCS] Remove SparkSession constructor invocation in the example
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #40505:
URL: https://github.com/apache/spark/pull/40505#issuecomment-1478755239
Merged to master and branch-3.4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org