You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/03/31 03:32:42 UTC
[GitHub] [spark] morvenhuang opened a new pull request #36020: SPARK-38707 Allow user to insert into only certain columns of a table
morvenhuang opened a new pull request #36020:
URL: https://github.com/apache/spark/pull/36020
### What changes were proposed in this pull request?
Update the column number check logic for INSERT INTO statement, avoid checking all columns of the table when user has specified a column list in the statement.
### Why are the changes needed?
When running INSERT INTO statement, it's quite common that user wants to insert only certain columns, especially when the rest columns have default value.
Currently, spark allows user to specify column list for INSERT INTO statement only when the column list contains all columns of the table. If user does not provided a completed list of column, it will result in an AnalysisException.
This patch allows user to insert into only certain columns of the table, which will help when excuting INSERT INTO, especially when excuting them on RDBMS.
### Does this PR introduce _any_ user-facing change?
Yes.
### How was this patch tested?
New test case added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #36020: [SPARK-38707][SQL] Allow user to insert into only certain columns of a table
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #36020:
URL: https://github.com/apache/spark/pull/36020#issuecomment-1085419342
@morvenhuang it would be great to assess this further if you add some references of other DBMSes that support this syntax. Does Hive support this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] morvenhuang commented on pull request #36020: [SPARK-38707][SQL] Allow user to insert into only certain columns of a table
Posted by GitBox <gi...@apache.org>.
morvenhuang commented on pull request #36020:
URL: https://github.com/apache/spark/pull/36020#issuecomment-1085305857
> Can one of the admins verify this patch?
There're some failed test, I'm looking into it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #36020: [SPARK-38707][SQL] Allow user to insert into only certain columns of a table
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #36020:
URL: https://github.com/apache/spark/pull/36020#issuecomment-1086748663
cc @dtenedor FYI. Seems like actually it's already implemented with `spark.sql.defaultColumn.useNullsForMissingDefautValues` configuration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] morvenhuang commented on pull request #36020: [SPARK-38707][SQL] Allow user to insert into only certain columns of a table
Posted by GitBox <gi...@apache.org>.
morvenhuang commented on pull request #36020:
URL: https://github.com/apache/spark/pull/36020#issuecomment-1085504436
>
@HyukjinKwon Hi Hyukjin, thank you for your time. I believe this is a SQL-92 standard, most RDBMSes I've known support inserting only certain columns of a table, like MySQL/Oracle/MS SQL SERVER/Teradata. And yes, Hive also supports this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #36020: [SPARK-38707][SQL] Allow user to insert into only certain columns of a table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #36020:
URL: https://github.com/apache/spark/pull/36020#issuecomment-1084784667
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org