You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by misutoth <gi...@git.apache.org> on 2018/09/23 20:57:06 UTC
[GitHub] spark pull request #22532: [SPARK-20845][SQL] Support specification of colum...
GitHub user misutoth opened a pull request:
https://github.com/apache/spark/pull/22532
[SPARK-20845][SQL] Support specification of column names in INSERT INTO command.
## What changes were proposed in this pull request?
One can specify a list of columns for an INSERT INTO command. The columns shall be listed in parenthesis just following the table name. Query columns are then matched to this very same order.
```
scala> sql("CREATE TABLE t (s string, i int)")
scala> sql("INSERT INTO t values ('first', 1)")
scala> sql("INSERT INTO t (i, s) values (2, 'second')")
scala> sql("SELECT * FROM t").show
+------+---+
| s| i|
+------+---+
| first| 1|
|second| 2|
+------+---+
scala>
```
In the above example the _second_ insertion utilizes the new functionality. The number and its associated string is given in reverse order `(2, 'second')` according to the column list specified for the table `(i, s)`. The result can be seen at the end of the command list. Intermediate output of the commands are omitted for the sake of brevity.
## How was this patch tested?
InsertSuite (both in source and in hive sub-packages) were extended with tests exercising specification of column names listing in INSERT INTO commands.
Also ran the above sample, and ran tests in `sql`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/misutoth/spark insert-into-columns
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22532.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22532
----
commit 1dda672d336b906ecc133f468435b4cf38859e2d
Author: Mihaly Toth <mi...@...>
Date: 2018-03-20T06:13:01Z
[SPARK-20845][SQL] Support specification of column names in INSERT INTO command.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22532: [SPARK-20845][SQL] Support specification of column names...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22532
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22532: [SPARK-20845][SQL] Support specification of column names...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22532
**[Test build #97006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97006/testReport)** for PR 22532 at commit [`1dda672`](https://github.com/apache/spark/commit/1dda672d336b906ecc133f468435b4cf38859e2d).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22532: [SPARK-20845][SQL] Support specification of column names...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22532
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22532: [SPARK-20845][SQL] Support specification of column names...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22532
ok to test
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22532: [SPARK-20845][SQL] Support specification of column names...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22532
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97006/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22532: [SPARK-20845][SQL] Support specification of column names...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22532
**[Test build #97006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97006/testReport)** for PR 22532 at commit [`1dda672`](https://github.com/apache/spark/commit/1dda672d336b906ecc133f468435b4cf38859e2d).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22532: [SPARK-20845][SQL] Support specification of column names...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22532
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22532: [SPARK-20845][SQL] Support specification of column names...
Posted by misutoth <gi...@git.apache.org>.
Github user misutoth commented on the issue:
https://github.com/apache/spark/pull/22532
Many thanks for the feedback. I will list the test scenarios that I had in mind and collected while I implemented this item.
And sorry about the failure, seems like I did not rerun all the tests in my last step... For example when the same field is queried multiple times it is not handled properly. I will fix them also ...
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22532: [SPARK-20845][SQL] Support specification of column names...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22532
Thanks for submitting the PR! I quickly scan the code changes. It sounds like the general direction is right but the quality is not ready.
I would suggest to write the test plan before doing the code review. Could you try your best to write down what we should test for supporting this feature? Both negative and positive cases.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22532: [SPARK-20845][SQL] Support specification of column names...
Posted by misutoth <gi...@git.apache.org>.
Github user misutoth commented on the issue:
https://github.com/apache/spark/pull/22532
@janewangfb, @gatorsmile could you please possibly review this change?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22532: [SPARK-20845][SQL] Support specification of column names...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22532
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org