You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "hiboyang (via GitHub)" <gi...@apache.org> on 2023/06/02 06:03:17 UTC
[GitHub] [spark-connect-go] hiboyang opened a new pull request, #6: Add init Spark Connect Go files
hiboyang opened a new pull request, #6:
URL: https://github.com/apache/spark-connect-go/pull/6
### What changes were proposed in this pull request?
This pull request is to add a small Spark Connect Go client example and prototype.
JIRA: https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-43351
### Why are the changes needed?
Spark Connect was released in Spark 3.4.0. There is no Go client yet. Better to have a Go client so Spark Connect could be used by Go programmer.
### Does this PR introduce _any_ user-facing change?
Yes. User will be able to use Go to write Spark Connect application. A very simple example in Go looks like following:
```
func main() {
remote := "localhost:15002"
spark, _ := sql.SparkSession.Builder.Remote(remote).Build()
defer spark.Stop()
df, _ := spark.Sql("select 'apple' as word, 123 as count union all select 'orange' as word, 456 as count")
df.Show(100, false)
}
```
### How was this patch tested?
Manually tested by running the example Go code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark-connect-go] HyukjinKwon commented on a diff in pull request #6: [SPARK-43351] Add init Spark Connect Go files
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #6:
URL: https://github.com/apache/spark-connect-go/pull/6#discussion_r1214239650
##########
.gitignore:
##########
@@ -21,3 +21,6 @@ internal/generated.out
# Ignore Coverage Files
coverage*
+
+# Ignore IDE files
+.idea/
Review Comment:
```suggestion
.idea/
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark-connect-go] hiboyang commented on pull request #6: [SPARK-43351] Add init Spark Connect Go files
Posted by "hiboyang (via GitHub)" <gi...@apache.org>.
hiboyang commented on PR #6:
URL: https://github.com/apache/spark-connect-go/pull/6#issuecomment-1573972644
> For a quick pro tip, if you add your email `bo_yang6@apple.com` to your GitHub account, your contribution will be properly visible in GitHub ,e.g., at https://github.com/apache/spark-connect-go/graphs/contributors
Cool, thanks for the nice tip! Added my email in github just now :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark-connect-go] HyukjinKwon commented on pull request #6: [SPARK-43351] Add init Spark Connect Go files
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #6:
URL: https://github.com/apache/spark-connect-go/pull/6#issuecomment-1573636642
For a quick pro tip, if you add your email `bo_yang6@apple.com` to your GitHub account, your contribution will be properly visible in GitHub ,e.g., at https://github.com/apache/spark-connect-go/graphs/contributors
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark-connect-go] HyukjinKwon commented on pull request #6: [SPARK-43351] Add init Spark Connect Go files
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #6:
URL: https://github.com/apache/spark-connect-go/pull/6#issuecomment-1573567321
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark-connect-go] HyukjinKwon closed pull request #6: [SPARK-43351] Add init Spark Connect Go files
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #6: [SPARK-43351] Add init Spark Connect Go files
URL: https://github.com/apache/spark-connect-go/pull/6
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark-connect-go] hiboyang commented on pull request #6: Add init Spark Connect Go files
Posted by "hiboyang (via GitHub)" <gi...@apache.org>.
hiboyang commented on PR #6:
URL: https://github.com/apache/spark-connect-go/pull/6#issuecomment-1573202463
Hi @grundprinzip, I copied my code from [previous PR](https://github.com/apache/spark/pull/41036) to here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org