You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/03/18 17:52:30 UTC
[GitHub] [spark] Eric5553 opened a new pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Eric5553 opened a new pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952
### What changes were proposed in this pull request?
Hive 2.3+ supports `getTablesByType` API, which is a precondition to implement SHOW VIEWS in HiveExternalCatalog. Currently, without this API, we can not get hive tables with type `HiveTableType.VIRTUAL_VIEW` directly.
This PR add `getTablesByType` in `HiveShim`. For those hive versions don't support this API, `UnsupportedOperationException` will be thrown.
Since the JDK11 related hive fix is not released yet(https://issues.apache.org/jira/browse/SPARK-29245), manual tests against hive 2.3.7-SNAPSHOT is done by issuing `SHOW VIEWS` command.
https://github.com/apache/spark/pull/27897
### Why are the changes needed?
This API will provide better usability and performance if we want to get a list of hiveTables with specific type. For example `HiveTableType.VIRTUAL_VIEW` corresponding to `CatalogTableType.VIEW`.
### Does this PR introduce any user-facing change?
No, this is a support function.
### How was this patch tested?
Add tests in VersionsSuite and manually run JDK11 test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r395450798
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with Logging {
conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 1000L
}
+ override def getTablesByType(
+ hive: Hive,
+ dbName: String,
+ pattern: String,
+ tableType: TableType): Seq[String] = {
+ throw new UnsupportedOperationException("Hive 2.2 and lower versions don't support " +
+ "getTablesByType. Please use Hive 2.3 or higher version.")
Review comment:
We had better keep this `HiveShim` layer simple. +1 for @Eric5553 's suggestion to implement the fallback in `HiveClientImpl` layer.
cc @gatorsmile
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600851152
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120001/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601195975
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24772/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r396041969
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala
##########
@@ -405,12 +415,16 @@ class VersionsSuite extends SparkFunSuite with Logging {
try {
client.dropTable("default", tableName = "temporary", ignoreIfNotExists = false,
purge = true)
+ client.dropTable("default", tableName = "view1", ignoreIfNotExists = false,
+ purge = true)
Review comment:
nit. Usually, we had better have separate `try .. catch` statements.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r395450798
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with Logging {
conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 1000L
}
+ override def getTablesByType(
+ hive: Hive,
+ dbName: String,
+ pattern: String,
+ tableType: TableType): Seq[String] = {
+ throw new UnsupportedOperationException("Hive 2.2 and lower versions don't support " +
+ "getTablesByType. Please use Hive 2.3 or higher version.")
Review comment:
We had better keep this `HiveShim` layer simple. +1 for @Eric5553 's suggestion to implement the fallback in `HiveClient` layer.
cc @gatorsmile
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601868307
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602110548
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24850/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601751435
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24818/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602122211
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601293137
**[Test build #120057 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120057/testReport)** for PR 27952 at commit [`90bd6d5`](https://github.com/apache/spark/commit/90bd6d58fa1b68592a3f861620d16281673b30f6).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602110548
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24850/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600777693
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600948761
@maropu Thanks for helping maintain these PRs :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601294926
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120057/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600776871
JDK11 tested manually with following settings:
```
Hive 2.3.6 Metastore on JDK8
Hive 2.3.7-SNAPSHOT library build from Hive 2.3 branch
Spark build with Hive 2.3.7-SNAPSHOT on jdk-11.0.6
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601823549
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601946941
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601199228
**[Test build #120057 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120057/testReport)** for PR 27952 at commit [`90bd6d5`](https://github.com/apache/spark/commit/90bd6d58fa1b68592a3f861620d16281673b30f6).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r395451362
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -755,6 +755,22 @@ private[hive] class HiveClientImpl(
client.getTablesByPattern(dbName, pattern).asScala
}
+ override def listTablesByType(
+ dbName: String,
+ pattern: String,
+ tableType: CatalogTableType): Seq[String] = withHiveState {
+ try {
+ // Try with Hive API getTablesByType first, it's supported from Hive 2.3.
Review comment:
`Hive 2.3` -> `Hive 2.3+`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602125335
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
maropu commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601946171
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r394765212
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with Logging {
conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 1000L
}
+ override def getTablesByType(
+ hive: Hive,
+ dbName: String,
+ pattern: String,
+ tableType: TableType): Seq[String] = {
+ throw new UnsupportedOperationException("Hive 2.2 and lower versions don't support " +
+ "getTablesByType. Please use Hive 2.3 or higher version.")
Review comment:
@Eric5553, do you mean we'll support `SHOW VIEWS` only with Hive 2.3 in Spark side?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601751425
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600777139
**[Test build #120001 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120001/testReport)** for PR 27952 at commit [`2ef13ec`](https://github.com/apache/spark/commit/2ef13ece94aafc5d48cf97fb87e311ed5e250067).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600851152
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120001/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601195152
**[Test build #120055 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120055/testReport)** for PR 27952 at commit [`0a550b1`](https://github.com/apache/spark/commit/0a550b17ff24e43144d841ac449b510ef1c297cf).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r394767919
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with Logging {
conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 1000L
}
+ override def getTablesByType(
+ hive: Hive,
+ dbName: String,
+ pattern: String,
+ tableType: TableType): Seq[String] = {
+ throw new UnsupportedOperationException("Hive 2.2 and lower versions don't support " +
+ "getTablesByType. Please use Hive 2.3 or higher version.")
Review comment:
Sure, that's a mismatch. Updated the description, thanks :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601551296
Retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
maropu commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600888875
cc: @dongjoon-hyun @cloud-fan
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601195975
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24772/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602121925
**[Test build #120134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120134/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601552196
**[Test build #120080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120080/testReport)** for PR 27952 at commit [`90bd6d5`](https://github.com/apache/spark/commit/90bd6d58fa1b68592a3f861620d16281673b30f6).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602106872
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24848/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602110546
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601946624
**[Test build #120119 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120119/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601284785
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601294912
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r395056474
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with Logging {
conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 1000L
}
+ override def getTablesByType(
+ hive: Hive,
+ dbName: String,
+ pattern: String,
+ tableType: TableType): Seq[String] = {
+ throw new UnsupportedOperationException("Hive 2.2 and lower versions don't support " +
+ "getTablesByType. Please use Hive 2.3 or higher version.")
Review comment:
The fallback logic needs to call `HiveClient.getTablesByName`, which got a lot of logic in `HiveClient`. Thus I think it needs to handle the fallback logic in `HiveClient` instead of `HiveShim`. WDYT @maropu @cloud-fan @HyukjinKwon , thanks!
Also I renamed the `HiveClient.listViews` API to `HiveClient.listTablesByType` to provide a more common interface.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601751435
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24818/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601199919
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601199228
**[Test build #120057 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120057/testReport)** for PR 27952 at commit [`90bd6d5`](https://github.com/apache/spark/commit/90bd6d58fa1b68592a3f861620d16281673b30f6).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601826963
**[Test build #120111 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120111/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601195959
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600851147
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601294926
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120057/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 edited a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 edited a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600776871
JDK11 tested manually with following settings:
```
Hive 2.3.6 Metastore on JDK8
Hive 2.3.7-SNAPSHOT library build from source of Hive 2.3 branch
Spark build with Hive 2.3.7-SNAPSHOT on jdk-11.0.6
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601823566
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24824/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r394766305
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with Logging {
conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 1000L
}
+ override def getTablesByType(
+ hive: Hive,
+ dbName: String,
+ pattern: String,
+ tableType: TableType): Seq[String] = {
+ throw new UnsupportedOperationException("Hive 2.2 and lower versions don't support " +
+ "getTablesByType. Please use Hive 2.3 or higher version.")
Review comment:
Okay, so it's still possible to look up via `listTables`. Can you fix the PR description? seems not it's a precondition but just for better performance
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601199919
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601946946
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24832/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601561877
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601284797
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120055/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601199931
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24774/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601751425
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600777139
**[Test build #120001 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120001/testReport)** for PR 27952 at commit [`2ef13ec`](https://github.com/apache/spark/commit/2ef13ece94aafc5d48cf97fb87e311ed5e250067).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601946624
**[Test build #120119 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120119/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601561823
**[Test build #120080 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120080/testReport)** for PR 27952 at commit [`90bd6d5`](https://github.com/apache/spark/commit/90bd6d58fa1b68592a3f861620d16281673b30f6).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600777693
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601819555
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r395451175
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClient.scala
##########
@@ -61,6 +61,13 @@ private[hive] trait HiveClient {
/** Returns the names of tables in the given database that matches the given pattern. */
def listTables(dbName: String, pattern: String): Seq[String]
+ /** Returns the names of tables with specific tableType in the given database that matches
+ * the given pattern. */
Review comment:
```scala
/**
* Returns the names of tables with specific tableType in the given database that matches
* the given pattern.
*/
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601195959
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602110339
**[Test build #120136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120136/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602106872
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24848/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601750649
**[Test build #120104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120104/testReport)** for PR 27952 at commit [`52e2b43`](https://github.com/apache/spark/commit/52e2b432763a692524ceb66c6a8cc75af63c8cd7).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r394810540
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with Logging {
conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 1000L
}
+ override def getTablesByType(
+ hive: Hive,
+ dbName: String,
+ pattern: String,
+ tableType: TableType): Seq[String] = {
+ throw new UnsupportedOperationException("Hive 2.2 and lower versions don't support " +
+ "getTablesByType. Please use Hive 2.3 or higher version.")
Review comment:
Can't we do the fallback here? It's better to make the low-level API completed, e.g. `getTablesByType` always sucess, but may be optimized with Hive 2.3+
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601823549
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r394815124
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with Logging {
conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 1000L
}
+ override def getTablesByType(
+ hive: Hive,
+ dbName: String,
+ pattern: String,
+ tableType: TableType): Seq[String] = {
+ throw new UnsupportedOperationException("Hive 2.2 and lower versions don't support " +
+ "getTablesByType. Please use Hive 2.3 or higher version.")
Review comment:
Previously I thought `HiveShim` is a thin layer close to Hive API. Yea, it should be better if we handle it in low-level API. I'll make the change here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602125341
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120136/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601552405
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24797/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601946946
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24832/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601868307
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602106589
Retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 commented on a change in pull request #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r395701753
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -755,6 +755,22 @@ private[hive] class HiveClientImpl(
client.getTablesByPattern(dbName, pattern).asScala
}
+ override def listTablesByType(
+ dbName: String,
+ pattern: String,
+ tableType: CatalogTableType): Seq[String] = withHiveState {
+ try {
+ // Try with Hive API getTablesByType first, it's supported from Hive 2.3.
Review comment:
Yea, sure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601818428
**[Test build #120104 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120104/testReport)** for PR 27952 at commit [`52e2b43`](https://github.com/apache/spark/commit/52e2b432763a692524ceb66c6a8cc75af63c8cd7).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601284797
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120055/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600851147
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602106870
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602125335
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601552399
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601946941
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601533788
Thank you for making a dedicated one, @Eric5553 .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602107943
**[Test build #120134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120134/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601199931
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24774/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602107943
**[Test build #120134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120134/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602122211
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601284785
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601561882
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120080/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601552405
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24797/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601561882
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120080/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 commented on a change in pull request #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r395701285
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClient.scala
##########
@@ -61,6 +61,13 @@ private[hive] trait HiveClient {
/** Returns the names of tables in the given database that matches the given pattern. */
def listTables(dbName: String, pattern: String): Seq[String]
+ /** Returns the names of tables with specific tableType in the given database that matches
+ * the given pattern. */
Review comment:
I see, thanks for the review!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600777711
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24721/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600850531
**[Test build #120001 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120001/testReport)** for PR 27952 at commit [`2ef13ec`](https://github.com/apache/spark/commit/2ef13ece94aafc5d48cf97fb87e311ed5e250067).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601552196
**[Test build #120080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120080/testReport)** for PR 27952 at commit [`90bd6d5`](https://github.com/apache/spark/commit/90bd6d58fa1b68592a3f861620d16281673b30f6).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601868315
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120111/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602109444
Retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601868315
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120111/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601971507
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120119/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602122216
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120134/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601971507
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120119/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602122216
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120134/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601819565
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120104/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601823566
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24824/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r394815124
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with Logging {
conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 1000L
}
+ override def getTablesByType(
+ hive: Hive,
+ dbName: String,
+ pattern: String,
+ tableType: TableType): Seq[String] = {
+ throw new UnsupportedOperationException("Hive 2.2 and lower versions don't support " +
+ "getTablesByType. Please use Hive 2.3 or higher version.")
Review comment:
I thought `HiveShim` is a thin layer close to Hive API. Yea, it should be better if we handle it in low-level API. I'll make the change here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601868167
**[Test build #120111 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120111/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601750649
**[Test build #120104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120104/testReport)** for PR 27952 at commit [`52e2b43`](https://github.com/apache/spark/commit/52e2b432763a692524ceb66c6a8cc75af63c8cd7).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601195152
**[Test build #120055 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120055/testReport)** for PR 27952 at commit [`0a550b1`](https://github.com/apache/spark/commit/0a550b17ff24e43144d841ac449b510ef1c297cf).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601819555
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601294912
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602110546
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601826963
**[Test build #120111 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120111/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601283602
**[Test build #120055 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120055/testReport)** for PR 27952 at commit [`0a550b1`](https://github.com/apache/spark/commit/0a550b17ff24e43144d841ac449b510ef1c297cf).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601552399
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r395056474
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with Logging {
conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 1000L
}
+ override def getTablesByType(
+ hive: Hive,
+ dbName: String,
+ pattern: String,
+ tableType: TableType): Seq[String] = {
+ throw new UnsupportedOperationException("Hive 2.2 and lower versions don't support " +
+ "getTablesByType. Please use Hive 2.3 or higher version.")
Review comment:
The fallback logic needs to call `HiveClient.getTablesByName`, which got a lot of logic in `HiveClient`. Thus I think it needs to handle the fallback logic in `HiveClient` instead of `HiveShim`. 0a550b17ff24e43144d841ac449b510ef1c297cf WDYT @maropu @cloud-fan @HyukjinKwon , thanks!
Also I renamed the `HiveClient.listViews` API to `HiveClient.listTablesByType` to provide a more common interface.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602110339
**[Test build #120136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120136/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601005930
@HyukjinKwon Per the comments https://github.com/apache/spark/pull/27897#issuecomment-600964599, I updated the PR description as a common HiveClient API enhancement.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601971261
**[Test build #120119 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120119/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602106870
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601819565
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120104/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602125341
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120136/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601971504
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Eric5553 commented on a change in pull request #27952:
[SPARK-31184][SQL] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#discussion_r394766333
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -363,6 +370,15 @@ private[client] class Shim_v0_12 extends Shim with Logging {
conf.getIntVar(HiveConf.ConfVars.METASTORE_CLIENT_CONNECT_RETRY_DELAY) * 1000L
}
+ override def getTablesByType(
+ hive: Hive,
+ dbName: String,
+ pattern: String,
+ tableType: TableType): Seq[String] = {
+ throw new UnsupportedOperationException("Hive 2.2 and lower versions don't support " +
+ "getTablesByType. Please use Hive 2.3 or higher version.")
Review comment:
As @cloud-fan suggested in https://github.com/apache/spark/pull/27897#discussion_r394391663, for versions before Hive 2.3, I'll implement the fallback solution using `listTables` and filter the result in PR https://github.com/apache/spark/pull/27897.
Thanks for the review! @HyukjinKwon
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27952:
[SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API
of Hive Client
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27952: [SPARK-31184][SQL][test-hadoop3.2][test-java11] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-602125051
**[Test build #120136 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120136/testReport)** for PR 27952 at commit [`ff3777f`](https://github.com/apache/spark/commit/ff3777faf81079fb7a5f66c84037b3121f8a846a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601971504
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952: [SPARK-31184][SQL]
Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-600777711
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24721/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27952:
[SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27952: [SPARK-31184][SQL][test-hive1.2] Support getTablesByType API of Hive Client
URL: https://github.com/apache/spark/pull/27952#issuecomment-601561877
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org