You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/03/06 19:31:56 UTC
[GitHub] [spark] gatorsmile opened a new pull request #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
gatorsmile opened a new pull request #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839
### What changes were proposed in this pull request?
Based on the discussion in the mailing list [[Proposal] Modification to Spark's Semantic Versioning Policy](http://apache-spark-developers-list.1001551.n3.nabble.com/Proposal-Modification-to-Spark-s-Semantic-Versioning-Policy-td28938.html) , this PR is to add back the following APIs whose maintenance cost are relatively small.
- SQLContext.applySchema
- SQLContext.parquetFile
- SQLContext.jsonFile
- SQLContext.jsonRDD
- SQLContext.load
- SQLContext.jdbc
Note: This PR has not added back the corresponding changes made in SparkR.
### Why are the changes needed?
Avoid breaking the APIs that are commonly used.
### Does this PR introduce any user-facing change?
Adding back the APIs that were removed in 3.0 branch does not introduce the user-facing changes, because Spark 3.0 has not been released.
### How was this patch tested?
TODO: add a new test suite for these APIs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#discussion_r389180456
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala
##########
@@ -611,6 +611,289 @@ class SQLContext private[sql](val sparkSession: SparkSession)
sessionState.catalog.listTables(databaseName).map(_.table).toArray
}
+ ////////////////////////////////////////////////////////////////////////////
+ ////////////////////////////////////////////////////////////////////////////
+ // Deprecated methods
+ ////////////////////////////////////////////////////////////////////////////
+ ////////////////////////////////////////////////////////////////////////////
+
+ /**
+ * @deprecated As of 1.3.0, replaced by `createDataFrame()`.
+ */
+ @deprecated("Use createDataFrame instead.", "1.3.0")
Review comment:
Hi, @marmbrus and @srowen .
Do you really need to keep `1.3.0` API?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603065115
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120233/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-595925935
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24211/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-597800753
Also, cc @marmbrus .
And, cc @rxin since he is the release manager for 3.0.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-602990953
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24946/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603064898
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-604843265
Thanks! Merged to master/3.0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27839: [SPARK-31086][SQL] Add
Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603064793
**[Test build #120243 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120243/testReport)** for PR 27839 at commit [`de7811d`](https://github.com/apache/spark/commit/de7811d7644e44d110a3ef3e25edbc51d146fdda).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-596015565
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119482/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gatorsmile closed pull request #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
gatorsmile closed pull request #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-596015565
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119482/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-596015556
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27839: [SPARK-31086][SQL] Add
Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603499405
**[Test build #4999 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4999/testReport)** for PR 27839 at commit [`de7811d`](https://github.com/apache/spark/commit/de7811d7644e44d110a3ef3e25edbc51d146fdda).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-595998740
Yes, is it simpler to just list the things being proposed to change and why (i.e. any data or arguments to support it) first? It does seem like a whole lot is being added back, which is at odds with what I understand to be the conclusion of the last thread on this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27839: [SPARK-31086][SQL] Add
Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603064799
**[Test build #120233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120233/testReport)** for PR 27839 at commit [`2b1dcb6`](https://github.com/apache/spark/commit/2b1dcb6fcc45b97a75c6ab15eff93621637e5345).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class DeprecatedAPISuite extends QueryTest with SharedSparkSession `
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603064898
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-602990948
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603064911
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120243/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-602990521
**[Test build #120233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120233/testReport)** for PR 27839 at commit [`2b1dcb6`](https://github.com/apache/spark/commit/2b1dcb6fcc45b97a75c6ab15eff93621637e5345).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-595928526
**[Test build #119482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119482/testReport)** for PR 27839 at commit [`d3b5845`](https://github.com/apache/spark/commit/d3b58450dc3ca465f8a8515ed45b21640c99a520).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603331399
**[Test build #4999 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4999/testReport)** for PR 27839 at commit [`de7811d`](https://github.com/apache/spark/commit/de7811d7644e44d110a3ef3e25edbc51d146fdda).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-596015161
**[Test build #119482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119482/testReport)** for PR 27839 at commit [`d3b5845`](https://github.com/apache/spark/commit/d3b58450dc3ca465f8a8515ed45b21640c99a520).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603065106
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-595925926
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603042884
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
gatorsmile commented on a change in pull request #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#discussion_r395435576
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala
##########
@@ -611,6 +611,289 @@ class SQLContext private[sql](val sparkSession: SparkSession)
sessionState.catalog.listTables(databaseName).map(_.table).toArray
}
+ ////////////////////////////////////////////////////////////////////////////
+ ////////////////////////////////////////////////////////////////////////////
+ // Deprecated methods
+ ////////////////////////////////////////////////////////////////////////////
+ ////////////////////////////////////////////////////////////////////////////
+
+ /**
+ * @deprecated As of 1.3.0, replaced by `createDataFrame()`.
+ */
+ @deprecated("Use createDataFrame instead.", "1.3.0")
Review comment:
At least these Spark books are still using this API in their examples, although the first two claims they are updated using Spark 2.x
- https://www.amazon.com/Mastering-Apache-Spark-2-x-DeepLearning4j/dp/1786462745/ref=sr_1_4?keywords=mastering+Apache+Spark&qid=1584678721&s=books&sr=1-4
- https://www.manning.com/books/spark-in-action
- http://shop.oreilly.com/product/0636920028512.do [first edition]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-595928526
**[Test build #119482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119482/testReport)** for PR 27839 at commit [`d3b5845`](https://github.com/apache/spark/commit/d3b58450dc3ca465f8a8515ed45b21640c99a520).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] marmbrus commented on a change in pull request #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
marmbrus commented on a change in pull request #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#discussion_r391196792
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala
##########
@@ -611,6 +611,289 @@ class SQLContext private[sql](val sparkSession: SparkSession)
sessionState.catalog.listTables(databaseName).map(_.table).toArray
}
+ ////////////////////////////////////////////////////////////////////////////
+ ////////////////////////////////////////////////////////////////////////////
+ // Deprecated methods
+ ////////////////////////////////////////////////////////////////////////////
+ ////////////////////////////////////////////////////////////////////////////
+
+ /**
+ * @deprecated As of 1.3.0, replaced by `createDataFrame()`.
+ */
+ @deprecated("Use createDataFrame instead.", "1.3.0")
Review comment:
The rubric says nothing about how old the deprecation warning is on purpose. It says we should think about usage.
@gatorsmile do you have any reason to believe these are commonly used functions?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603041462
CC @dongjoon-hyun @marmbrus @rxin @srowen This PR is ready for review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603042894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24957/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-595925935
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24211/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-596015556
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-602990953
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24946/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603064911
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120243/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#discussion_r389182283
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala
##########
@@ -611,6 +611,289 @@ class SQLContext private[sql](val sparkSession: SparkSession)
sessionState.catalog.listTables(databaseName).map(_.table).toArray
}
+ ////////////////////////////////////////////////////////////////////////////
+ ////////////////////////////////////////////////////////////////////////////
+ // Deprecated methods
+ ////////////////////////////////////////////////////////////////////////////
+ ////////////////////////////////////////////////////////////////////////////
+
+ /**
+ * @deprecated As of 1.3.0, replaced by `createDataFrame()`.
+ */
+ @deprecated("Use createDataFrame instead.", "1.3.0")
Review comment:
I agree; I also don't feel strongly about it as there isn't much overhead to keeping it, beyond API noise.
@gatorsmile may have some additional info or data about this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603042402
**[Test build #120243 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120243/testReport)** for PR 27839 at commit [`de7811d`](https://github.com/apache/spark/commit/de7811d7644e44d110a3ef3e25edbc51d146fdda).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-602990948
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603042884
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839:
[SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-25821][SQL][FOLLOW-UP] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-595925926
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603042894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24957/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27839: [SPARK-31086][SQL] Add
Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-602990521
**[Test build #120233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120233/testReport)** for PR 27839 at commit [`2b1dcb6`](https://github.com/apache/spark/commit/2b1dcb6fcc45b97a75c6ab15eff93621637e5345).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27839: [SPARK-31086][SQL] Add
Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603042402
**[Test build #120243 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120243/testReport)** for PR 27839 at commit [`de7811d`](https://github.com/apache/spark/commit/de7811d7644e44d110a3ef3e25edbc51d146fdda).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27839: [SPARK-31086][SQL]
Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603065106
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27839: [SPARK-31086][SQL] Add
Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603331399
**[Test build #4999 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4999/testReport)** for PR 27839 at commit [`de7811d`](https://github.com/apache/spark/commit/de7811d7644e44d110a3ef3e25edbc51d146fdda).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27839:
[SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27839: [SPARK-31086][SQL] Add Back the Deprecated SQLContext methods
URL: https://github.com/apache/spark/pull/27839#issuecomment-603065115
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120233/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org