You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/20 06:55:56 UTC
[GitHub] [spark] imback82 opened a new pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
imback82 opened a new pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
For the following:
```
CREATE TABLE t USING json AS SELECT 1 AS i
SELECT * FROM spark_catalog.t
```
`spark_catalog.t` is resolved to `spark_catalog.default.t` assuming the current namespace is `default`. However, this is not consistent with V2 behavior where the namespace must be specified if the catalog name is provided. This PR proposes to fix this inconsistency.
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
To be consistent with V2 table naming scheme in SQL commands.
### Does this PR introduce any user-facing change?
<!--
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If no, write 'No'.
-->
Yes, now the user has to specify the namespace if the catalog name is provided. For example,
```
SELECT * FROM spark_catalog.t # Will throw AnalysisException with 'Session catalog cannot have an empty namespace: spark_catalog.t'
SELECT * FROM spark_catalog.default.t # OK
```
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
Added new tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589512042
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383086146
##########
File path: sql/core/src/test/resources/sql-tests/results/show-create-table.sql.out
##########
@@ -44,7 +44,7 @@ SHOW CREATE TABLE tbl
-- !query schema
struct<createtab_stmt:string>
-- !query output
-CREATE TABLE `tbl` (
+CREATE TABLE `default`.`tbl` (
Review comment:
not related to this PR but a future improvement to v2 commands: since we resolve the catalog and tables during the analysis phase, it would be better to display the fully qualified table name(include catalog name) in EXPLAIN, to let users know which table exactly was picked by the command.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382392250
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -449,8 +454,17 @@ class ResolveSessionCatalog(
partitionSpec)
case ShowColumnsStatement(tbl, ns) =>
+ if (ns.isDefined && ns.get.length > 1) {
+ throw new AnalysisException(
+ s"Namespace name should have only one part if specified: ${ns.get.quoted}")
+ }
+ val nameParts = if (ns.isDefined && tbl.length == 1) {
+ ns.get ++ tbl
+ } else {
+ tbl
+ }
val sql = "SHOW COLUMNS"
- val v1TableName = parseV1Table(tbl, sql).asTableIdentifier
+ val v1TableName = parseTempViewOrV1Table(nameParts, sql).asTableIdentifier
Review comment:
`ShowColumnsCommand` works with temp views as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590204453
**[Test build #118848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118848/testReport)** for PR 27642 at commit [`b43890a`](https://github.com/apache/spark/commit/b43890a6bdcdeabfee4e6b926e1531e9221e156f).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590220359
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382962850
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/CachedTableSuite.scala
##########
@@ -102,18 +102,18 @@ class CachedTableSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
}
test("uncache of nonexistent tables") {
- val expectedErrorMsg = "Table or view not found: nonexistentTable"
+ val expectedErrorMsg = "Table or view not found:"
// make sure table doesn't exist
var e = intercept[AnalysisException](spark.table("nonexistentTable")).getMessage
- assert(e.contains(expectedErrorMsg))
+ assert(e.contains(s"$expectedErrorMsg nonexistentTable"))
Review comment:
`spark.table` just creates `UnresolvedRelation(multipartIdentifier)`, so it doesn't add current namespace.
Perhaps, should we retaine the original multipart names in `UnresolvedRelation`, etc. and use it inside `CheckAnalysis`? For `SELECT * FROM tbl`, what is a more desirable message, `Table or view not found: tbl` or `Table or view not found: default.tbl`?
Note that this PR always adds the current namespace to the identifier and postgres seems to use the original identifier given.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590100062
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[WIP][SPARK-30885][SQL] V1 table name should be fully qualified if catalog
name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [WIP][SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589493292
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382391949
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -415,6 +416,10 @@ class ResolveSessionCatalog(
partition)
case ShowCreateTableStatement(tbl, asSerde) if !asSerde =>
+ if (isTempView(tbl)) {
Review comment:
This check is moved from `ShowCreateTableCommand`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382961153
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -257,9 +257,10 @@ class ResolveSessionCatalog(
case v1Table: V1Table =>
DescribeColumnCommand(tbl.asTableIdentifier, colNameParts, isExtended)
}.getOrElse {
- if (isTempView(tbl)) {
Review comment:
updated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590175375
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23596/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590029261
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118827/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590204604
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118848/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590321128
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118858/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yuchenhuo commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
yuchenhuo commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382865667
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -449,8 +454,17 @@ class ResolveSessionCatalog(
partitionSpec)
case ShowColumnsStatement(tbl, ns) =>
+ if (ns.isDefined && ns.get.length > 1) {
+ throw new AnalysisException(
+ s"Namespace name should have only one part if specified: ${ns.get.quoted}")
+ }
+ val nameParts = if (ns.isDefined && tbl.length == 1) {
Review comment:
Can we perhaps add a comment here why this `tbl.length == 1` is necessary?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590022399
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590024846
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383087768
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSessionCatalogSuite.scala
##########
@@ -39,23 +40,26 @@ class DataSourceV2SQLSessionCatalogSuite
override protected def verifyTable(tableName: String, expected: DataFrame): Unit = {
checkAnswer(spark.table(tableName), expected)
checkAnswer(sql(s"SELECT * FROM $tableName"), expected)
- checkAnswer(sql(s"SELECT * FROM default.$tableName"), expected)
checkAnswer(sql(s"TABLE $tableName"), expected)
}
override def getTableMetadata(tableName: String): Table = {
val v2Catalog = spark.sessionState.catalogManager.currentCatalog
val nameParts = spark.sessionState.sqlParser.parseMultipartIdentifier(tableName)
- v2Catalog.asInstanceOf[TableCatalog]
- .loadTable(Identifier.of(Array.empty, nameParts.last))
+ val ident = if (v2Catalog.name == SESSION_CATALOG_NAME) {
+ Identifier.of(nameParts.init.toArray, nameParts.last)
+ } else {
+ Identifier.of(Array.empty, nameParts.last)
Review comment:
Can't we write `Identifier.of(nameParts.init.toArray, nameParts.last)` for the else branch as well?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382962397
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -659,14 +669,7 @@ class ResolveSessionCatalog(
object SessionCatalogAndTable {
def unapply(nameParts: Seq[String]): Option[(CatalogPlugin, Seq[String])] = nameParts match {
case SessionCatalogAndIdentifier(catalog, ident) =>
- if (nameParts.length == 1) {
Review comment:
OK. I will do it as a follow up. (I couldn't just remove the check because of `SPARK-30799: temp view name can't contain catalog name`, but I will think about it as a follow up).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382402352
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2SessionCatalog.scala
##########
@@ -171,8 +171,6 @@ class V2SessionCatalog(catalog: SessionCatalog, conf: SQLConf)
ident.namespace match {
case Array(db) =>
TableIdentifier(ident.name, Some(db))
- case Array() =>
Review comment:
shall we give a better error message when database is not present?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590559997
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118881/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589494913
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590218884
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382395068
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala
##########
@@ -2121,23 +2120,40 @@ class DataSourceV2SQLSuite
withTable("t") {
sql("CREATE TABLE t USING json AS SELECT 1 AS i")
checkAnswer(sql("select * from t"), Row(1))
- checkAnswer(sql("select * from spark_catalog.t"), Row(1))
checkAnswer(sql("select * from spark_catalog.default.t"), Row(1))
}
}
+ test("SPARK-30885: v1 table name should be fully qualified") {
+ def run(): Unit = {
Review comment:
fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590449142
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-588724739
**[Test build #118701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118701/testReport)** for PR 27642 at commit [`018e6f6`](https://github.com/apache/spark/commit/018e6f6da7c698e52a219cd520a811e2a1e8d3c1).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590321128
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118858/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589494739
**[Test build #118751 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118751/testReport)** for PR 27642 at commit [`fea7909`](https://github.com/apache/spark/commit/fea790947e3f213a89d9c1bbdfcc86b564058ed9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590024779
**[Test build #118827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118827/testReport)** for PR 27642 at commit [`1cf8907`](https://github.com/apache/spark/commit/1cf8907f8406aac40e004313936158e7a5cb15c8).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589511970
**[Test build #118751 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118751/testReport)** for PR 27642 at commit [`fea7909`](https://github.com/apache/spark/commit/fea790947e3f213a89d9c1bbdfcc86b564058ed9).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590023217
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23576/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590029261
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118827/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-588643848
**[Test build #118701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118701/testReport)** for PR 27642 at commit [`018e6f6`](https://github.com/apache/spark/commit/018e6f6da7c698e52a219cd520a811e2a1e8d3c1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383092112
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/connector/TestV2SessionCatalogBase.scala
##########
@@ -43,7 +44,7 @@ private[connector] trait TestV2SessionCatalogBase[T <: Table] extends Delegating
protected def fullIdentifier(ident: Identifier): Identifier = {
Review comment:
Good catch! Removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382962850
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/CachedTableSuite.scala
##########
@@ -102,18 +102,18 @@ class CachedTableSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
}
test("uncache of nonexistent tables") {
- val expectedErrorMsg = "Table or view not found: nonexistentTable"
+ val expectedErrorMsg = "Table or view not found:"
// make sure table doesn't exist
var e = intercept[AnalysisException](spark.table("nonexistentTable")).getMessage
- assert(e.contains(expectedErrorMsg))
+ assert(e.contains(s"$expectedErrorMsg nonexistentTable"))
Review comment:
`spark.table` just creates `UnresolvedRelation(multipartIdentifier)`, so it doesn't add current namespace.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590028534
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590028537
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118825/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589512046
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118751/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589494917
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23502/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589496694
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23503/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r381825279
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/LookupCatalog.scala
##########
@@ -106,24 +107,29 @@ private[sql] trait LookupCatalog extends Logging {
def unapply(nameParts: Seq[String]): Option[(CatalogPlugin, Identifier)] = {
assert(nameParts.nonEmpty)
- if (nameParts.length == 1) {
- Some((currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head)))
+ val (catalog, ident) = if (nameParts.length == 1) {
+ (currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head))
} else if (nameParts.head.equalsIgnoreCase(globalTempDB)) {
// Conceptually global temp views are in a special reserved catalog. However, the v2 catalog
// API does not support view yet, and we have to use v1 commands to deal with global temp
// views. To simplify the implementation, we put global temp views in a special namespace
// in the session catalog. The special namespace has higher priority during name resolution.
// For example, if the name of a custom catalog is the same with `GLOBAL_TEMP_DATABASE`,
// this custom catalog can't be accessed.
- Some((catalogManager.v2SessionCatalog, nameParts.asIdentifier))
+ (catalogManager.v2SessionCatalog, nameParts.asIdentifier)
} else {
try {
- Some((catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier))
+ (catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier)
} catch {
case _: CatalogNotFoundException =>
- Some((currentCatalog, nameParts.asIdentifier))
+ (currentCatalog, nameParts.asIdentifier)
}
}
+ if (CatalogV2Util.isSessionCatalog(catalog) && ident.namespace.isEmpty) {
Review comment:
I think `V2SessionCatalog` should stop filling the default database. It should assume the input identifier is the final identifier like other catalogs, and fail it doesn't have database part. Then we don't need to do the check here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-588725351
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r381826963
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/LookupCatalog.scala
##########
@@ -106,24 +107,29 @@ private[sql] trait LookupCatalog extends Logging {
def unapply(nameParts: Seq[String]): Option[(CatalogPlugin, Identifier)] = {
assert(nameParts.nonEmpty)
- if (nameParts.length == 1) {
- Some((currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head)))
+ val (catalog, ident) = if (nameParts.length == 1) {
+ (currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head))
} else if (nameParts.head.equalsIgnoreCase(globalTempDB)) {
// Conceptually global temp views are in a special reserved catalog. However, the v2 catalog
// API does not support view yet, and we have to use v1 commands to deal with global temp
// views. To simplify the implementation, we put global temp views in a special namespace
// in the session catalog. The special namespace has higher priority during name resolution.
// For example, if the name of a custom catalog is the same with `GLOBAL_TEMP_DATABASE`,
// this custom catalog can't be accessed.
- Some((catalogManager.v2SessionCatalog, nameParts.asIdentifier))
+ (catalogManager.v2SessionCatalog, nameParts.asIdentifier)
} else {
try {
- Some((catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier))
+ (catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier)
} catch {
case _: CatalogNotFoundException =>
- Some((currentCatalog, nameParts.asIdentifier))
+ (currentCatalog, nameParts.asIdentifier)
}
}
+ if (CatalogV2Util.isSessionCatalog(catalog) && ident.namespace.isEmpty) {
Review comment:
Good point, let me explore this route. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590100065
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23585/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589543852
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589496401
**[Test build #118752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118752/testReport)** for PR 27642 at commit [`95d2297`](https://github.com/apache/spark/commit/95d22972ba15767557b22674eedce6e6889666e3).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590449153
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23630/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590175179
**[Test build #118848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118848/testReport)** for PR 27642 at commit [`b43890a`](https://github.com/apache/spark/commit/b43890a6bdcdeabfee4e6b926e1531e9221e156f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590448359
**[Test build #118881 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118881/testReport)** for PR 27642 at commit [`eb0ebf1`](https://github.com/apache/spark/commit/eb0ebf1990c126d352c61f757d68686faf9c7f4b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590220365
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23607/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383278863
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -415,6 +404,10 @@ class ResolveSessionCatalog(
partition)
case ShowCreateTableStatement(tbl, asSerde) if !asSerde =>
+ if (isTempView(tbl)) {
+ throw new AnalysisException(
Review comment:
actually, can we just call `parseTempViewOrV1Table` here?
e.g.
```
val name = parseTempViewOrV1Table...
ShowCreateTableCommand(name.asTableIdentifier, ...)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590023217
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23576/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589543555
**[Test build #118752 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118752/testReport)** for PR 27642 at commit [`95d2297`](https://github.com/apache/spark/commit/95d22972ba15767557b22674eedce6e6889666e3).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382960538
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2SessionCatalog.scala
##########
@@ -171,8 +171,6 @@ class V2SessionCatalog(catalog: SessionCatalog, conf: SQLConf)
ident.namespace match {
case Array(db) =>
TableIdentifier(ident.name, Some(db))
- case Array() =>
Review comment:
Yes, good idea!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589494917
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23502/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589542538
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118747/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589542538
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118747/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590175369
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590559232
**[Test build #118881 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118881/testReport)** for PR 27642 at commit [`eb0ebf1`](https://github.com/apache/spark/commit/eb0ebf1990c126d352c61f757d68686faf9c7f4b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yuchenhuo commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
yuchenhuo commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383047812
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -449,8 +454,17 @@ class ResolveSessionCatalog(
partitionSpec)
case ShowColumnsStatement(tbl, ns) =>
+ if (ns.isDefined && ns.get.length > 1) {
+ throw new AnalysisException(
+ s"Namespace name should have only one part if specified: ${ns.get.quoted}")
+ }
+ val nameParts = if (ns.isDefined && tbl.length == 1) {
Review comment:
Actually, the part confuses me is when would `tbl.length != 1`. Is it because sometimes we would keep the namespace in the table name? It just feels strange that ns might not represent the actual namespace for the table.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r381817006
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/LookupCatalog.scala
##########
@@ -106,24 +107,29 @@ private[sql] trait LookupCatalog extends Logging {
def unapply(nameParts: Seq[String]): Option[(CatalogPlugin, Identifier)] = {
assert(nameParts.nonEmpty)
- if (nameParts.length == 1) {
- Some((currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head)))
+ val (catalog, ident) = if (nameParts.length == 1) {
+ (currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head))
} else if (nameParts.head.equalsIgnoreCase(globalTempDB)) {
// Conceptually global temp views are in a special reserved catalog. However, the v2 catalog
// API does not support view yet, and we have to use v1 commands to deal with global temp
// views. To simplify the implementation, we put global temp views in a special namespace
// in the session catalog. The special namespace has higher priority during name resolution.
// For example, if the name of a custom catalog is the same with `GLOBAL_TEMP_DATABASE`,
// this custom catalog can't be accessed.
- Some((catalogManager.v2SessionCatalog, nameParts.asIdentifier))
+ (catalogManager.v2SessionCatalog, nameParts.asIdentifier)
} else {
try {
- Some((catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier))
+ (catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier)
} catch {
case _: CatalogNotFoundException =>
- Some((currentCatalog, nameParts.asIdentifier))
+ (currentCatalog, nameParts.asIdentifier)
}
}
+ if (CatalogV2Util.isSessionCatalog(catalog) && ident.namespace.isEmpty) {
Review comment:
@cloud-fan For a session catalog, I could make this assumption that the namespace is required, right? I looked at `CatalogManager` that uses `v1SessionCatalog` for setting current namespace if the current catalog is a session catalog; and `v1SessionCatalog` requires the namespace (database) to already exist.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r381825279
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/LookupCatalog.scala
##########
@@ -106,24 +107,29 @@ private[sql] trait LookupCatalog extends Logging {
def unapply(nameParts: Seq[String]): Option[(CatalogPlugin, Identifier)] = {
assert(nameParts.nonEmpty)
- if (nameParts.length == 1) {
- Some((currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head)))
+ val (catalog, ident) = if (nameParts.length == 1) {
+ (currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head))
} else if (nameParts.head.equalsIgnoreCase(globalTempDB)) {
// Conceptually global temp views are in a special reserved catalog. However, the v2 catalog
// API does not support view yet, and we have to use v1 commands to deal with global temp
// views. To simplify the implementation, we put global temp views in a special namespace
// in the session catalog. The special namespace has higher priority during name resolution.
// For example, if the name of a custom catalog is the same with `GLOBAL_TEMP_DATABASE`,
// this custom catalog can't be accessed.
- Some((catalogManager.v2SessionCatalog, nameParts.asIdentifier))
+ (catalogManager.v2SessionCatalog, nameParts.asIdentifier)
} else {
try {
- Some((catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier))
+ (catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier)
} catch {
case _: CatalogNotFoundException =>
- Some((currentCatalog, nameParts.asIdentifier))
+ (currentCatalog, nameParts.asIdentifier)
}
}
+ if (CatalogV2Util.isSessionCatalog(catalog) && ident.namespace.isEmpty) {
Review comment:
I think `V2SessionCatalog` should stop filling the default database. It should assume the input identifier is the final identifier like other catalogs, and fail if identifier doesn't have database part. Then we don't need to do the check here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590022777
**[Test build #118825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118825/testReport)** for PR 27642 at commit [`9a4671c`](https://github.com/apache/spark/commit/9a4671ce46bd036f33fdfdb4ad5f03ea5074ceb0).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590028537
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118825/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-588645554
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590123129
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118836/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590023525
**[Test build #118826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118826/testReport)** for PR 27642 at commit [`4e80fd2`](https://github.com/apache/spark/commit/4e80fd287ba27b27257992782a4488c172bc5420).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590029238
**[Test build #118827 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118827/testReport)** for PR 27642 at commit [`1cf8907`](https://github.com/apache/spark/commit/1cf8907f8406aac40e004313936158e7a5cb15c8).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589493022
**[Test build #118747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118747/testReport)** for PR 27642 at commit [`29c88f1`](https://github.com/apache/spark/commit/29c88f176d8687454bf54bac3c1153a67641209c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589494739
**[Test build #118751 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118751/testReport)** for PR 27642 at commit [`fea7909`](https://github.com/apache/spark/commit/fea790947e3f213a89d9c1bbdfcc86b564058ed9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382401496
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -257,9 +257,10 @@ class ResolveSessionCatalog(
case v1Table: V1Table =>
DescribeColumnCommand(tbl.asTableIdentifier, colNameParts, isExtended)
}.getOrElse {
- if (isTempView(tbl)) {
Review comment:
can we follow how we deal with `UncacheTableStatement`? Basically just call `parseTempViewOrV1Table`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590093444
**[Test build #118835 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118835/testReport)** for PR 27642 at commit [`ef83ba1`](https://github.com/apache/spark/commit/ef83ba1f4c00e661ac7131016854f61879210e0d).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590023216
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590449153
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23630/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590085727
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23584/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589543852
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590320279
**[Test build #118858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118858/testReport)** for PR 27642 at commit [`b43890a`](https://github.com/apache/spark/commit/b43890a6bdcdeabfee4e6b926e1531e9221e156f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590122964
**[Test build #118836 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118836/testReport)** for PR 27642 at commit [`bcd9a65`](https://github.com/apache/spark/commit/bcd9a6591aa27f3ca22d8176b66e37b9460ca353).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590028994
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589496694
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23503/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382391902
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -396,7 +397,7 @@ class ResolveSessionCatalog(
}
case AnalyzeColumnStatement(tbl, columnNames, allColumns) =>
- val v1TableName = parseV1Table(tbl, "ANALYZE TABLE")
+ val v1TableName = parseTempViewOrV1Table(tbl, "ANALYZE TABLE")
Review comment:
`AnalyzeColumnCommand` actually supports temp view.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590204599
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590123129
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118836/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383278863
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -415,6 +404,10 @@ class ResolveSessionCatalog(
partition)
case ShowCreateTableStatement(tbl, asSerde) if !asSerde =>
+ if (isTempView(tbl)) {
+ throw new AnalysisException(
Review comment:
actually, can we just call `parseTempViewOrV1Table` here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590220359
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-588643848
**[Test build #118701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118701/testReport)** for PR 27642 at commit [`018e6f6`](https://github.com/apache/spark/commit/018e6f6da7c698e52a219cd520a811e2a1e8d3c1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590028994
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383087496
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/connector/TestV2SessionCatalogBase.scala
##########
@@ -43,7 +44,7 @@ private[connector] trait TestV2SessionCatalogBase[T <: Table] extends Delegating
protected def fullIdentifier(ident: Identifier): Identifier = {
Review comment:
We can remove this method completely.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590093469
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118835/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590123127
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589512046
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118751/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590093466
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382394370
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
##########
@@ -1085,47 +1085,42 @@ case class ShowCreateTableCommand(table: TableIdentifier)
override def run(sparkSession: SparkSession): Seq[Row] = {
val catalog = sparkSession.sessionState.catalog
- if (catalog.isTemporaryTable(table)) {
Review comment:
The change is only removing this `if` block, but diff is bad.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590028997
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118826/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-588645583
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23452/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590219927
**[Test build #118858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118858/testReport)** for PR 27642 at commit [`b43890a`](https://github.com/apache/spark/commit/b43890a6bdcdeabfee4e6b926e1531e9221e156f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382402104
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -659,14 +669,7 @@ class ResolveSessionCatalog(
object SessionCatalogAndTable {
def unapply(nameParts: Seq[String]): Option[(CatalogPlugin, Seq[String])] = nameParts match {
case SessionCatalogAndIdentifier(catalog, ident) =>
- if (nameParts.length == 1) {
Review comment:
so we are going to remove the hack in `TempViewOrV1Table` in a followup PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590099938
**[Test build #118836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118836/testReport)** for PR 27642 at commit [`bcd9a65`](https://github.com/apache/spark/commit/bcd9a6591aa27f3ca22d8176b66e37b9460ca353).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590024848
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23577/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590024846
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590685443
thanks, merging to master/3.0!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r381823564
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala
##########
@@ -2121,23 +2120,40 @@ class DataSourceV2SQLSuite
withTable("t") {
sql("CREATE TABLE t USING json AS SELECT 1 AS i")
checkAnswer(sql("select * from t"), Row(1))
- checkAnswer(sql("select * from spark_catalog.t"), Row(1))
checkAnswer(sql("select * from spark_catalog.default.t"), Row(1))
}
}
+ test("SPARK-30885: v1 table name should be fully qualified") {
+ def run(): Unit = {
Review comment:
nit: `assertWrongTableIdent`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-588645583
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23452/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589493022
**[Test build #118747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118747/testReport)** for PR 27642 at commit [`29c88f1`](https://github.com/apache/spark/commit/29c88f176d8687454bf54bac3c1153a67641209c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590559988
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589496689
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590022399
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590093466
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382961099
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
##########
@@ -1085,47 +1085,42 @@ case class ShowCreateTableCommand(table: TableIdentifier)
override def run(sparkSession: SparkSession): Seq[Row] = {
val catalog = sparkSession.sessionState.catalog
- if (catalog.isTemporaryTable(table)) {
Review comment:
reverted.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590029260
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589542266
**[Test build #118747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118747/testReport)** for PR 27642 at commit [`29c88f1`](https://github.com/apache/spark/commit/29c88f176d8687454bf54bac3c1153a67641209c).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590220365
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23607/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r381822779
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/LookupCatalog.scala
##########
@@ -106,24 +107,29 @@ private[sql] trait LookupCatalog extends Logging {
def unapply(nameParts: Seq[String]): Option[(CatalogPlugin, Identifier)] = {
assert(nameParts.nonEmpty)
- if (nameParts.length == 1) {
- Some((currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head)))
+ val (catalog, ident) = if (nameParts.length == 1) {
+ (currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head))
} else if (nameParts.head.equalsIgnoreCase(globalTempDB)) {
// Conceptually global temp views are in a special reserved catalog. However, the v2 catalog
// API does not support view yet, and we have to use v1 commands to deal with global temp
// views. To simplify the implementation, we put global temp views in a special namespace
// in the session catalog. The special namespace has higher priority during name resolution.
// For example, if the name of a custom catalog is the same with `GLOBAL_TEMP_DATABASE`,
// this custom catalog can't be accessed.
- Some((catalogManager.v2SessionCatalog, nameParts.asIdentifier))
+ (catalogManager.v2SessionCatalog, nameParts.asIdentifier)
} else {
try {
- Some((catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier))
+ (catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier)
} catch {
case _: CatalogNotFoundException =>
- Some((currentCatalog, nameParts.asIdentifier))
+ (currentCatalog, nameParts.asIdentifier)
}
}
+ if (CatalogV2Util.isSessionCatalog(catalog) && ident.namespace.isEmpty) {
Review comment:
`CatalogAndIdentifier` should just focus on extracting catalog and identifier.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590175375
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23596/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589542532
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590028973
**[Test build #118826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118826/testReport)** for PR 27642 at commit [`4e80fd2`](https://github.com/apache/spark/commit/4e80fd287ba27b27257992782a4488c172bc5420).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383092066
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSessionCatalogSuite.scala
##########
@@ -39,23 +40,26 @@ class DataSourceV2SQLSessionCatalogSuite
override protected def verifyTable(tableName: String, expected: DataFrame): Unit = {
checkAnswer(spark.table(tableName), expected)
checkAnswer(sql(s"SELECT * FROM $tableName"), expected)
- checkAnswer(sql(s"SELECT * FROM default.$tableName"), expected)
checkAnswer(sql(s"TABLE $tableName"), expected)
}
override def getTableMetadata(tableName: String): Table = {
val v2Catalog = spark.sessionState.catalogManager.currentCatalog
val nameParts = spark.sessionState.sqlParser.parseMultipartIdentifier(tableName)
- v2Catalog.asInstanceOf[TableCatalog]
- .loadTable(Identifier.of(Array.empty, nameParts.last))
+ val ident = if (v2Catalog.name == SESSION_CATALOG_NAME) {
+ Identifier.of(nameParts.init.toArray, nameParts.last)
+ } else {
+ Identifier.of(Array.empty, nameParts.last)
Review comment:
You are right. Updated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590024779
**[Test build #118827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118827/testReport)** for PR 27642 at commit [`1cf8907`](https://github.com/apache/spark/commit/1cf8907f8406aac40e004313936158e7a5cb15c8).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590023525
**[Test build #118826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118826/testReport)** for PR 27642 at commit [`4e80fd2`](https://github.com/apache/spark/commit/4e80fd287ba27b27257992782a4488c172bc5420).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590085549
**[Test build #118835 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118835/testReport)** for PR 27642 at commit [`ef83ba1`](https://github.com/apache/spark/commit/ef83ba1f4c00e661ac7131016854f61879210e0d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yuchenhuo commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
yuchenhuo commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382866793
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -396,7 +397,7 @@ class ResolveSessionCatalog(
}
case AnalyzeColumnStatement(tbl, columnNames, allColumns) =>
- val v1TableName = parseV1Table(tbl, "ANALYZE TABLE")
+ val v1TableName = parseTempViewOrV1Table(tbl, "ANALYZE TABLE")
Review comment:
Just want to check if I understand this correctly. This reason why we need to make this change is that `parseV1Table` would accidentally add currentName as the namespace for a temp view which is incorrect because temp view shouldn't have a namespace?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383399234
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSessionCatalogSuite.scala
##########
@@ -19,13 +19,14 @@ package org.apache.spark.sql.connector
import org.apache.spark.sql.{DataFrame, SaveMode}
import org.apache.spark.sql.connector.catalog.{Identifier, Table, TableCatalog}
+import org.apache.spark.sql.connector.catalog.CatalogManager.SESSION_CATALOG_NAME
class DataSourceV2SQLSessionCatalogSuite
extends InsertIntoTests(supportsDynamicOverwrite = true, includeSQLOnlyTests = true)
with AlterTableTests
with SessionCatalogTest[InMemoryTable, InMemoryTableSessionCatalog] {
- override protected val catalogAndNamespace = ""
+ override protected val catalogAndNamespace = "default."
Review comment:
Yea makes sense. Thanks for pointing this out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589494913
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-588725351
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590028997
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118826/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589542532
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642:
[WIP][SPARK-30885][SQL] V1 table name should be fully qualified if catalog
name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [WIP][SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589493297
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23498/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590029260
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589512042
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590321116
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590175179
**[Test build #118848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118848/testReport)** for PR 27642 at commit [`b43890a`](https://github.com/apache/spark/commit/b43890a6bdcdeabfee4e6b926e1531e9221e156f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r381825279
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/LookupCatalog.scala
##########
@@ -106,24 +107,29 @@ private[sql] trait LookupCatalog extends Logging {
def unapply(nameParts: Seq[String]): Option[(CatalogPlugin, Identifier)] = {
assert(nameParts.nonEmpty)
- if (nameParts.length == 1) {
- Some((currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head)))
+ val (catalog, ident) = if (nameParts.length == 1) {
+ (currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head))
} else if (nameParts.head.equalsIgnoreCase(globalTempDB)) {
// Conceptually global temp views are in a special reserved catalog. However, the v2 catalog
// API does not support view yet, and we have to use v1 commands to deal with global temp
// views. To simplify the implementation, we put global temp views in a special namespace
// in the session catalog. The special namespace has higher priority during name resolution.
// For example, if the name of a custom catalog is the same with `GLOBAL_TEMP_DATABASE`,
// this custom catalog can't be accessed.
- Some((catalogManager.v2SessionCatalog, nameParts.asIdentifier))
+ (catalogManager.v2SessionCatalog, nameParts.asIdentifier)
} else {
try {
- Some((catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier))
+ (catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier)
} catch {
case _: CatalogNotFoundException =>
- Some((currentCatalog, nameParts.asIdentifier))
+ (currentCatalog, nameParts.asIdentifier)
}
}
+ if (CatalogV2Util.isSessionCatalog(catalog) && ident.namespace.isEmpty) {
Review comment:
I think `V2SessionCatalog` should stop filling the default database. It should assume the input identifier is the final identifier and fail it doesn't have database part. Then we don't need to do the check here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-588725387
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118701/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590023216
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590024848
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23577/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590099938
**[Test build #118836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118836/testReport)** for PR 27642 at commit [`bcd9a65`](https://github.com/apache/spark/commit/bcd9a6591aa27f3ca22d8176b66e37b9460ca353).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590321116
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590559988
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382962230
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -449,8 +454,17 @@ class ResolveSessionCatalog(
partitionSpec)
case ShowColumnsStatement(tbl, ns) =>
+ if (ns.isDefined && ns.get.length > 1) {
+ throw new AnalysisException(
+ s"Namespace name should have only one part if specified: ${ns.get.quoted}")
+ }
+ val nameParts = if (ns.isDefined && tbl.length == 1) {
Review comment:
Added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590022777
**[Test build #118825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118825/testReport)** for PR 27642 at commit [`9a4671c`](https://github.com/apache/spark/commit/9a4671ce46bd036f33fdfdb4ad5f03ea5074ceb0).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r381824585
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/PlanResolutionSuite.scala
##########
@@ -128,7 +128,7 @@ class PlanResolutionSuite extends AnalysisTest {
}
})
when(manager.currentCatalog).thenReturn(v2SessionCatalog)
- when(manager.currentNamespace).thenReturn(Array.empty[String])
+ when(manager.currentNamespace).thenReturn(Array("default"))
Review comment:
why is this needed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-588725387
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118701/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642:
[WIP][SPARK-30885][SQL] V1 table name should be fully qualified if catalog
name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [WIP][SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589493292
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590123127
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-588645554
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383378881
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -415,6 +404,10 @@ class ResolveSessionCatalog(
partition)
case ShowCreateTableStatement(tbl, asSerde) if !asSerde =>
+ if (isTempView(tbl)) {
+ throw new AnalysisException(
Review comment:
OK.
That was my original implementation, but I did it this way because you will get `SHOW CREATE TABLE is only supported with temp views or v1 tables` then if you pass temp view, you get `SHOW CREATE TABLE is not supported on a temporary view`.
But I will just call `parseTempVieworV1Table` here. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589496689
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590022400
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23575/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590559997
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118881/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382394491
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2SessionCatalog.scala
##########
@@ -171,8 +171,6 @@ class V2SessionCatalog(catalog: SessionCatalog, conf: SQLConf)
ident.namespace match {
case Array(db) =>
TableIdentifier(ident.name, Some(db))
- case Array() =>
Review comment:
This is the actual change that removes filling the db automatically.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383281551
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSessionCatalogSuite.scala
##########
@@ -19,13 +19,14 @@ package org.apache.spark.sql.connector
import org.apache.spark.sql.{DataFrame, SaveMode}
import org.apache.spark.sql.connector.catalog.{Identifier, Table, TableCatalog}
+import org.apache.spark.sql.connector.catalog.CatalogManager.SESSION_CATALOG_NAME
class DataSourceV2SQLSessionCatalogSuite
extends InsertIntoTests(supportsDynamicOverwrite = true, includeSQLOnlyTests = true)
with AlterTableTests
with SessionCatalogTest[InMemoryTable, InMemoryTableSessionCatalog] {
- override protected val catalogAndNamespace = ""
+ override protected val catalogAndNamespace = "default."
Review comment:
thinking about this more. can we avoid changing it by updating the test cases that check error message?
I've already seen similar code, e.g. in `InsertIntoTests`
```
val tableName = if (catalogAndNamespace.isEmpty) s"default.$t1" else t1
assert(exc.getMessage.contains(s"Cannot write to '$tableName', too many data columns"))
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590100062
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589496401
**[Test build #118752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118752/testReport)** for PR 27642 at commit [`95d2297`](https://github.com/apache/spark/commit/95d22972ba15767557b22674eedce6e6889666e3).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590175369
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r381822407
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/LookupCatalog.scala
##########
@@ -106,24 +107,29 @@ private[sql] trait LookupCatalog extends Logging {
def unapply(nameParts: Seq[String]): Option[(CatalogPlugin, Identifier)] = {
assert(nameParts.nonEmpty)
- if (nameParts.length == 1) {
- Some((currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head)))
+ val (catalog, ident) = if (nameParts.length == 1) {
+ (currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head))
} else if (nameParts.head.equalsIgnoreCase(globalTempDB)) {
// Conceptually global temp views are in a special reserved catalog. However, the v2 catalog
// API does not support view yet, and we have to use v1 commands to deal with global temp
// views. To simplify the implementation, we put global temp views in a special namespace
// in the session catalog. The special namespace has higher priority during name resolution.
// For example, if the name of a custom catalog is the same with `GLOBAL_TEMP_DATABASE`,
// this custom catalog can't be accessed.
- Some((catalogManager.v2SessionCatalog, nameParts.asIdentifier))
+ (catalogManager.v2SessionCatalog, nameParts.asIdentifier)
} else {
try {
- Some((catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier))
+ (catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier)
} catch {
case _: CatalogNotFoundException =>
- Some((currentCatalog, nameParts.asIdentifier))
+ (currentCatalog, nameParts.asIdentifier)
}
}
+ if (CatalogV2Util.isSessionCatalog(catalog) && ident.namespace.isEmpty) {
Review comment:
or inside `ResolveSessionCatalog`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590085724
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590085549
**[Test build #118835 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118835/testReport)** for PR 27642 at commit [`ef83ba1`](https://github.com/apache/spark/commit/ef83ba1f4c00e661ac7131016854f61879210e0d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589543855
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118752/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590204599
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590449142
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590022400
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23575/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r381823606
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/LookupCatalog.scala
##########
@@ -106,24 +107,29 @@ private[sql] trait LookupCatalog extends Logging {
def unapply(nameParts: Seq[String]): Option[(CatalogPlugin, Identifier)] = {
assert(nameParts.nonEmpty)
- if (nameParts.length == 1) {
- Some((currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head)))
+ val (catalog, ident) = if (nameParts.length == 1) {
+ (currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head))
} else if (nameParts.head.equalsIgnoreCase(globalTempDB)) {
// Conceptually global temp views are in a special reserved catalog. However, the v2 catalog
// API does not support view yet, and we have to use v1 commands to deal with global temp
// views. To simplify the implementation, we put global temp views in a special namespace
// in the session catalog. The special namespace has higher priority during name resolution.
// For example, if the name of a custom catalog is the same with `GLOBAL_TEMP_DATABASE`,
// this custom catalog can't be accessed.
- Some((catalogManager.v2SessionCatalog, nameParts.asIdentifier))
+ (catalogManager.v2SessionCatalog, nameParts.asIdentifier)
} else {
try {
- Some((catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier))
+ (catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier)
} catch {
case _: CatalogNotFoundException =>
- Some((currentCatalog, nameParts.asIdentifier))
+ (currentCatalog, nameParts.asIdentifier)
}
}
+ if (CatalogV2Util.isSessionCatalog(catalog) && ident.namespace.isEmpty) {
Review comment:
The reason I put here is that `CatalogAndIdentifier` is also used here: https://github.com/apache/spark/blob/7c4ad6316e3fe4a58629771a99faaa986c866c0b/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriterV2.scala#L53-L57, which means I have to put the same check everywhere `CatalogAndIdentifier` is used?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590219927
**[Test build #118858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118858/testReport)** for PR 27642 at commit [`b43890a`](https://github.com/apache/spark/commit/b43890a6bdcdeabfee4e6b926e1531e9221e156f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r383054925
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -449,8 +454,17 @@ class ResolveSessionCatalog(
partitionSpec)
case ShowColumnsStatement(tbl, ns) =>
+ if (ns.isDefined && ns.get.length > 1) {
+ throw new AnalysisException(
+ s"Namespace name should have only one part if specified: ${ns.get.quoted}")
+ }
+ val nameParts = if (ns.isDefined && tbl.length == 1) {
Review comment:
Yea, you can do the following: `SHOW COLUMNS FROM db1.t1 IN db1`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382402286
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
##########
@@ -1085,47 +1085,42 @@ case class ShowCreateTableCommand(table: TableIdentifier)
override def run(sparkSession: SparkSession): Seq[Row] = {
val catalog = sparkSession.sessionState.catalog
- if (catalog.isTemporaryTable(table)) {
Review comment:
shall we keep the `if`? it doesn't hurt.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590204604
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118848/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590448359
**[Test build #118881 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118881/testReport)** for PR 27642 at commit [`eb0ebf1`](https://github.com/apache/spark/commit/eb0ebf1990c126d352c61f757d68686faf9c7f4b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590085727
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23584/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[WIP][SPARK-30885][SQL] V1 table name should be fully qualified if catalog
name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [WIP][SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589493297
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23498/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590028534
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27642: [SPARK-30885][SQL] V1
table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590028499
**[Test build #118825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118825/testReport)** for PR 27642 at commit [`9a4671c`](https://github.com/apache/spark/commit/9a4671ce46bd036f33fdfdb4ad5f03ea5074ceb0).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382962850
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/CachedTableSuite.scala
##########
@@ -102,18 +102,18 @@ class CachedTableSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
}
test("uncache of nonexistent tables") {
- val expectedErrorMsg = "Table or view not found: nonexistentTable"
+ val expectedErrorMsg = "Table or view not found:"
// make sure table doesn't exist
var e = intercept[AnalysisException](spark.table("nonexistentTable")).getMessage
- assert(e.contains(expectedErrorMsg))
+ assert(e.contains(s"$expectedErrorMsg nonexistentTable"))
Review comment:
`spark.table` just creates `UnresolvedRelation(multipartIdentifier)`, so it doesn't add current namespace.
Perhaps, should we retain the original multipart names somehow and use it inside `CheckAnalysis`? For `SELECT * FROM tbl`, what is a more desirable error message, `Table or view not found: tbl` or `Table or view not found: default.tbl`?
Note that this PR always adds the current namespace to the identifier and postgres seems to use the original identifier given.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382401496
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -257,9 +257,10 @@ class ResolveSessionCatalog(
case v1Table: V1Table =>
DescribeColumnCommand(tbl.asTableIdentifier, colNameParts, isExtended)
}.getOrElse {
- if (isTempView(tbl)) {
Review comment:
can we follow how we deal with `CacheTableStatement`? Basically just call `parseTempViewOrV1Table`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590093469
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118835/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-589543855
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118752/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27642: [SPARK-30885][SQL]
V1 table name should be fully qualified if catalog name is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590100065
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23585/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#issuecomment-590085724
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r381821285
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/LookupCatalog.scala
##########
@@ -106,24 +107,29 @@ private[sql] trait LookupCatalog extends Logging {
def unapply(nameParts: Seq[String]): Option[(CatalogPlugin, Identifier)] = {
assert(nameParts.nonEmpty)
- if (nameParts.length == 1) {
- Some((currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head)))
+ val (catalog, ident) = if (nameParts.length == 1) {
+ (currentCatalog, Identifier.of(catalogManager.currentNamespace, nameParts.head))
} else if (nameParts.head.equalsIgnoreCase(globalTempDB)) {
// Conceptually global temp views are in a special reserved catalog. However, the v2 catalog
// API does not support view yet, and we have to use v1 commands to deal with global temp
// views. To simplify the implementation, we put global temp views in a special namespace
// in the session catalog. The special namespace has higher priority during name resolution.
// For example, if the name of a custom catalog is the same with `GLOBAL_TEMP_DATABASE`,
// this custom catalog can't be accessed.
- Some((catalogManager.v2SessionCatalog, nameParts.asIdentifier))
+ (catalogManager.v2SessionCatalog, nameParts.asIdentifier)
} else {
try {
- Some((catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier))
+ (catalogManager.catalog(nameParts.head), nameParts.tail.asIdentifier)
} catch {
case _: CatalogNotFoundException =>
- Some((currentCatalog, nameParts.asIdentifier))
+ (currentCatalog, nameParts.asIdentifier)
}
}
+ if (CatalogV2Util.isSessionCatalog(catalog) && ident.namespace.isEmpty) {
Review comment:
shall we do this check inside `SessionCatalogAndIdentifier`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382395112
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/PlanResolutionSuite.scala
##########
@@ -128,7 +128,7 @@ class PlanResolutionSuite extends AnalysisTest {
}
})
when(manager.currentCatalog).thenReturn(v2SessionCatalog)
- when(manager.currentNamespace).thenReturn(Array.empty[String])
+ when(manager.currentNamespace).thenReturn(Array("default"))
Review comment:
no longer needed now. thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27642:
[SPARK-30885][SQL] V1 table name should be fully qualified if catalog name
is provided
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27642: [SPARK-30885][SQL] V1 table name should be fully qualified if catalog name is provided
URL: https://github.com/apache/spark/pull/27642#discussion_r382961077
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -396,7 +397,7 @@ class ResolveSessionCatalog(
}
case AnalyzeColumnStatement(tbl, columnNames, allColumns) =>
- val v1TableName = parseV1Table(tbl, "ANALYZE TABLE")
+ val v1TableName = parseTempViewOrV1Table(tbl, "ANALYZE TABLE")
Review comment:
yes, and this command supports temp views.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org