You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2023/01/13 19:47:28 UTC

[GitHub] [spark] techaddict opened a new pull request, #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv

techaddict opened a new pull request, #39559:
URL: https://github.com/apache/spark/pull/39559

   ### What changes were proposed in this pull request?
   This PR implements `DataFrameReader.csv` alias in Spark Connect.
   
   ### Why are the changes needed?
   For API feature parity.
   
   ### Does this PR introduce any user-facing change?
   This PR adds a user-facing API but Spark Connect has not been released yet.
   
   ### How was this patch tested?
   Unittest was added.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] techaddict commented on a diff in pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv

Posted by GitBox <gi...@apache.org>.
techaddict commented on code in PR #39559:
URL: https://github.com/apache/spark/pull/39559#discussion_r1070173868


##########
python/pyspark/sql/connect/readwriter.py:
##########
@@ -514,6 +586,7 @@ def _test() -> None:
         del pyspark.sql.connect.readwriter.DataFrameReader.load.__doc__
         del pyspark.sql.connect.readwriter.DataFrameReader.option.__doc__
         del pyspark.sql.connect.readwriter.DataFrameReader.text.__doc__
+        del pyspark.sql.connect.readwriter.DataFrameReader.csv.__doc__

Review Comment:
   It passed locally, was working against an older master branch.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] techaddict commented on pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv

Posted by GitBox <gi...@apache.org>.
techaddict commented on PR #39559:
URL: https://github.com/apache/spark/pull/39559#issuecomment-1382603008

   Updated the PR, tests will start passing once #39553 is merged, as its using `path: PathOrPaths`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HyukjinKwon commented on pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on PR #39559:
URL: https://github.com/apache/spark/pull/39559#issuecomment-1383121575

   Merged to master.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HyukjinKwon commented on pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on PR #39559:
URL: https://github.com/apache/spark/pull/39559#issuecomment-1382676645

   That PR is merged. Mind retriggering the test?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HyukjinKwon closed pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv

Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv
URL: https://github.com/apache/spark/pull/39559


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HyukjinKwon commented on a diff in pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on code in PR #39559:
URL: https://github.com/apache/spark/pull/39559#discussion_r1070151153


##########
python/pyspark/sql/connect/readwriter.py:
##########
@@ -514,6 +586,7 @@ def _test() -> None:
         del pyspark.sql.connect.readwriter.DataFrameReader.load.__doc__
         del pyspark.sql.connect.readwriter.DataFrameReader.option.__doc__
         del pyspark.sql.connect.readwriter.DataFrameReader.text.__doc__
+        del pyspark.sql.connect.readwriter.DataFrameReader.csv.__doc__

Review Comment:
   Hm, does the test fail?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org