You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2023/01/13 19:47:28 UTC
[GitHub] [spark] techaddict opened a new pull request, #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv
techaddict opened a new pull request, #39559:
URL: https://github.com/apache/spark/pull/39559
### What changes were proposed in this pull request?
This PR implements `DataFrameReader.csv` alias in Spark Connect.
### Why are the changes needed?
For API feature parity.
### Does this PR introduce any user-facing change?
This PR adds a user-facing API but Spark Connect has not been released yet.
### How was this patch tested?
Unittest was added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] techaddict commented on a diff in pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv
Posted by GitBox <gi...@apache.org>.
techaddict commented on code in PR #39559:
URL: https://github.com/apache/spark/pull/39559#discussion_r1070173868
##########
python/pyspark/sql/connect/readwriter.py:
##########
@@ -514,6 +586,7 @@ def _test() -> None:
del pyspark.sql.connect.readwriter.DataFrameReader.load.__doc__
del pyspark.sql.connect.readwriter.DataFrameReader.option.__doc__
del pyspark.sql.connect.readwriter.DataFrameReader.text.__doc__
+ del pyspark.sql.connect.readwriter.DataFrameReader.csv.__doc__
Review Comment:
It passed locally, was working against an older master branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] techaddict commented on pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv
Posted by GitBox <gi...@apache.org>.
techaddict commented on PR #39559:
URL: https://github.com/apache/spark/pull/39559#issuecomment-1382603008
Updated the PR, tests will start passing once #39553 is merged, as its using `path: PathOrPaths`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on PR #39559:
URL: https://github.com/apache/spark/pull/39559#issuecomment-1383121575
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on PR #39559:
URL: https://github.com/apache/spark/pull/39559#issuecomment-1382676645
That PR is merged. Mind retriggering the test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv
URL: https://github.com/apache/spark/pull/39559
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #39559: [SPARK-42011][CONNECT][PYTHON] Implement DataFrameReader.csv
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on code in PR #39559:
URL: https://github.com/apache/spark/pull/39559#discussion_r1070151153
##########
python/pyspark/sql/connect/readwriter.py:
##########
@@ -514,6 +586,7 @@ def _test() -> None:
del pyspark.sql.connect.readwriter.DataFrameReader.load.__doc__
del pyspark.sql.connect.readwriter.DataFrameReader.option.__doc__
del pyspark.sql.connect.readwriter.DataFrameReader.text.__doc__
+ del pyspark.sql.connect.readwriter.DataFrameReader.csv.__doc__
Review Comment:
Hm, does the test fail?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org