You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "hvanhovell (via GitHub)" <gi...@apache.org> on 2023/02/15 03:44:56 UTC

[GitHub] [spark] hvanhovell opened a new pull request, #40027: [SPARK-42441][CONNECT] Scala Client add Column APIs

hvanhovell opened a new pull request, #40027:
URL: https://github.com/apache/spark/pull/40027

   ### What changes were proposed in this pull request?
   This PR adds most Column APIs for the Spark Connect Scala Client.
   
   ### Why are the changes needed?
   We want the Scala Client to have API parity with the existing SparkSession/Dataset APIs.
   
   ### How was this patch tested?
   Golden files, and I added a test for local behavior.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] hvanhovell commented on pull request #40027: [SPARK-42441][CONNECT] Scala Client add Column APIs

Posted by "hvanhovell (via GitHub)" <gi...@apache.org>.
hvanhovell commented on PR #40027:
URL: https://github.com/apache/spark/pull/40027#issuecomment-1431838287

   merging.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #40027: [SPARK-42441][CONNECT] Scala Client add Column APIs

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #40027:
URL: https://github.com/apache/spark/pull/40027#issuecomment-1432288033

   FYI, bunch of Scala 2.13 tests fail (https://github.com/apache/spark/actions/runs/4187249065/jobs/7256849374#step:9:19944) should be fixed before cutting RC cc @xinrong-meng 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] LuciferYang commented on a diff in pull request #40027: [SPARK-42441][CONNECT] Scala Client add Column APIs

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #40027:
URL: https://github.com/apache/spark/pull/40027#discussion_r1107195659


##########
connector/connect/common/src/test/resources/query-tests/queries/column_when_otherwise.proto.bin:
##########
@@ -0,0 +1,14 @@
+�
+�Z��struct<id:bigint,a:int,b:double,d:struct<id:bigint,a:int,b:double>,e:array<int>,f:map<string,struct<id:bigint,a:int,b:double>>,g:string>OM

Review Comment:
   What are these unrecognized characters?
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] hvanhovell commented on a diff in pull request #40027: [SPARK-42441][CONNECT] Scala Client add Column APIs

Posted by "hvanhovell (via GitHub)" <gi...@apache.org>.
hvanhovell commented on code in PR #40027:
URL: https://github.com/apache/spark/pull/40027#discussion_r1107230074


##########
connector/connect/common/src/test/resources/query-tests/queries/column_when_otherwise.proto.bin:
##########
@@ -0,0 +1,14 @@
+�
+�Z��struct<id:bigint,a:int,b:double,d:struct<id:bigint,a:int,b:double>,e:array<int>,f:map<string,struct<id:bigint,a:int,b:double>>,g:string>OM

Review Comment:
   it is a binary file. It is not supposed to be readable.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on pull request #40027: [SPARK-42441][CONNECT] Scala Client add Column APIs

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #40027:
URL: https://github.com/apache/spark/pull/40027#issuecomment-1432574039

   Hi, @hvanhovell .
   Since we are close to RC1 cut, could you test Scala 2.13 before merging, please?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] hvanhovell closed pull request #40027: [SPARK-42441][CONNECT] Scala Client add Column APIs

Posted by "hvanhovell (via GitHub)" <gi...@apache.org>.
hvanhovell closed pull request #40027: [SPARK-42441][CONNECT] Scala Client add Column APIs
URL: https://github.com/apache/spark/pull/40027


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org