[GitHub] [spark] HyukjinKwon commented on a change in pull request #29023: local_connect_and_auth takes 2 arguments, 3 given

[GitBox <gi...@apache.org>]

HyukjinKwon commented on a change in pull request #29023:
URL: https://github.com/apache/spark/pull/29023#discussion_r451220383



##########
File path: python/pyspark/rdd.py
##########
@@ -141,7 +141,9 @@ def _parse_memory(s):
 
 
 def _load_from_socket(sock_info, serializer):
-    (sockfile, sock) = local_connect_and_auth(*sock_info)
+    port = sock_info[0]
+    auth_secret = sock_info[1]
+    (sockfile, sock) = local_connect_and_auth(port, auth_secret)

Review comment:
       This was correctly ported back at https://github.com/apache/spark/pull/25593. There's no such code you pointed out in Spark 2.4.4: https://github.com/apache/spark/blob/v2.4.4/python/pyspark/sql/dataframe.py#L2182.
   
   Are you using your own fork or mixing the Spark versions? Your error message seems from https://github.com/apache/spark/blob/v2.4.3/python/pyspark/sql/dataframe.py#L2179 which is Spark 2.4.3. Spark 2.4.3 does not have this change.
   
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org