You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/10/14 04:56:14 UTC
[GitHub] [spark] AngersZhuuuu opened a new pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
AngersZhuuuu opened a new pull request #34282:
URL: https://github.com/apache/spark/pull/34282
### What changes were proposed in this pull request?
Current doc about insert have too much repeated content, this pr is to merge two kind of docs.
### Why are the changes needed?
Merge related docs make it clear
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
Not need
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943163323
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48723/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943259449
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48730/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943082280
**[Test build #144243 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144243/testReport)** for PR 34282 at commit [`57ca213`](https://github.com/apache/spark/commit/57ca213e90284074ca283a0ce018029ed6243304).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943182152
**[Test build #144251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144251/testReport)** for PR 34282 at commit [`f00cf1d`](https://github.com/apache/spark/commit/f00cf1db67daac75df2829c84cc99826ad45fd1f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728673241
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format such as `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde` such as `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
Review comment:
```suggestion
2. Hive format using Hive `Serde` such as `ROW FORMAT row_format, `STORED AS file_format`. Hive support must be enabled to use this command.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943346568
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48734/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943276253
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48731/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943261021
**[Test build #144255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144255/testReport)** for PR 34282 at commit [`81971fe`](https://github.com/apache/spark/commit/81971fe77055ca3bb11dc99e86a528f7d370f0b9).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943246735
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48730/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943170841
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48723/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943227233
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144252/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943217321
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144251/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943090832
**[Test build #144243 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144243/testReport)** for PR 34282 at commit [`57ca213`](https://github.com/apache/spark/commit/57ca213e90284074ca283a0ce018029ed6243304).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943091134
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144243/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943070740
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48717/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943270247
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144255/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943082280
**[Test build #144243 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144243/testReport)** for PR 34282 at commit [`57ca213`](https://github.com/apache/spark/commit/57ca213e90284074ca283a0ce018029ed6243304).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728792664
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format: `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde`: `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
Review comment:
seems unnecessary to repeat the SQL syntax here. how about
```
... with the new values using either spark format or Hive Serde. Hive support must be enabled to use Hive Serde.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943217098
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48730/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943217769
**[Test build #144252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144252/testReport)** for PR 34282 at commit [`3811987`](https://github.com/apache/spark/commit/3811987710c3e8535a7cdf6ef90c042d2a549390).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-942969998
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144237/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943259449
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48730/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728675951
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format such as `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde` such as `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
+
+The inserted rows can be specified by value expressions or result from a query.
### Syntax
```sql
INSERT OVERWRITE [ LOCAL ] DIRECTORY [ directory_path ]
- USING file_format [ OPTIONS ( key = val [ , ... ] ) ]
+ { USING spark_file_format [ OPTIONS ( key = val [ , ... ] ) ] | [ ROW FORMAT row_format ] [ STORED AS hive_file_format ] }
{ VALUES ( { value | NULL } [ , ... ] ) [ , ( ... ) ] | query }
```
### Parameters
* **directory_path**
- Specifies the destination directory. It can also be specified in `OPTIONS` using `path`.
- The `LOCAL` keyword is used to specify that the directory is on the local file system.
+ Specifies the destination directory. The `LOCAL` keyword is used to specify that the directory is on the local file system.
+ In spark file format, it can also be specified in `OPTIONS` using `path`.
Review comment:
what if both `directory_path` and `path` option is specified?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728800280
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format: `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde`: `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
Review comment:
Looks good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943226981
**[Test build #144252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144252/testReport)** for PR 34282 at commit [`3811987`](https://github.com/apache/spark/commit/3811987710c3e8535a7cdf6ef90c042d2a549390).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-942969998
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144237/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728693728
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format such as `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde` such as `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
+
+The inserted rows can be specified by value expressions or result from a query.
### Syntax
```sql
INSERT OVERWRITE [ LOCAL ] DIRECTORY [ directory_path ]
- USING file_format [ OPTIONS ( key = val [ , ... ] ) ]
+ { USING spark_file_format [ OPTIONS ( key = val [ , ... ] ) ] | [ ROW FORMAT row_format ] [ STORED AS hive_file_format ] }
{ VALUES ( { value | NULL } [ , ... ] ) [ , ( ... ) ] | query }
```
### Parameters
* **directory_path**
- Specifies the destination directory. It can also be specified in `OPTIONS` using `path`.
- The `LOCAL` keyword is used to specify that the directory is on the local file system.
+ Specifies the destination directory. The `LOCAL` keyword is used to specify that the directory is on the local file system.
+ In spark file format, it can also be specified in `OPTIONS` using `path`.
Review comment:
updated `directory_path and `path` in OPTIONS should be specified one.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943270247
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144255/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943346568
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48734/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943080137
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48717/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943227233
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144252/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943261021
**[Test build #144255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144255/testReport)** for PR 34282 at commit [`81971fe`](https://github.com/apache/spark/commit/81971fe77055ca3bb11dc99e86a528f7d370f0b9).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728792824
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format such as `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde` such as `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
+
+The inserted rows can be specified by value expressions or result from a query.
### Syntax
```sql
INSERT OVERWRITE [ LOCAL ] DIRECTORY [ directory_path ]
- USING file_format [ OPTIONS ( key = val [ , ... ] ) ]
+ { USING spark_file_format [ OPTIONS ( key = val [ , ... ] ) ] | [ ROW FORMAT row_format ] [ STORED AS hive_file_format ] }
Review comment:
where do you mention it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #34282:
URL: https://github.com/apache/spark/pull/34282
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728673241
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format such as `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde` such as `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
Review comment:
```suggestion
2. Hive format using Hive `Serde`: `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-942969202
**[Test build #144237 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144237/testReport)** for PR 34282 at commit [`7fa046d`](https://github.com/apache/spark/commit/7fa046d61e3540f824a7b370c5a5622ce2ed396c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728676829
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format such as `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde` such as `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
+
+The inserted rows can be specified by value expressions or result from a query.
### Syntax
```sql
INSERT OVERWRITE [ LOCAL ] DIRECTORY [ directory_path ]
- USING file_format [ OPTIONS ( key = val [ , ... ] ) ]
+ { USING spark_file_format [ OPTIONS ( key = val [ , ... ] ) ] | [ ROW FORMAT row_format ] [ STORED AS hive_file_format ] }
Review comment:
`[ ROW FORMAT row_format ] [ STORED AS hive_file_format ]` are optional. What if neither of them is specified?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943182152
**[Test build #144251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144251/testReport)** for PR 34282 at commit [`f00cf1d`](https://github.com/apache/spark/commit/f00cf1db67daac75df2829c84cc99826ad45fd1f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943249409
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48731/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943191237
**[Test build #144251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144251/testReport)** for PR 34282 at commit [`f00cf1d`](https://github.com/apache/spark/commit/f00cf1db67daac75df2829c84cc99826ad45fd1f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728792664
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format: `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde`: `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
Review comment:
seems unnecessary to repeat the SQL syntax here. how about
```
... with the new values using either spark file format or Hive Serde. Hive support must be enabled to use Hive Serde.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728693960
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format such as `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde` such as `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
+
+The inserted rows can be specified by value expressions or result from a query.
### Syntax
```sql
INSERT OVERWRITE [ LOCAL ] DIRECTORY [ directory_path ]
- USING file_format [ OPTIONS ( key = val [ , ... ] ) ]
+ { USING spark_file_format [ OPTIONS ( key = val [ , ... ] ) ] | [ ROW FORMAT row_format ] [ STORED AS hive_file_format ] }
Review comment:
Have been introduce in `hive_file_format` section
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943405440
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-942957872
**[Test build #144237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144237/testReport)** for PR 34282 at commit [`7fa046d`](https://github.com/apache/spark/commit/7fa046d61e3540f824a7b370c5a5622ce2ed396c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-942957872
**[Test build #144237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144237/testReport)** for PR 34282 at commit [`7fa046d`](https://github.com/apache/spark/commit/7fa046d61e3540f824a7b370c5a5622ce2ed396c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-942993752
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48717/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943091134
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144243/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943294439
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48734/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-942953487
ping @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728849540
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -46,6 +57,18 @@ INSERT OVERWRITE [ LOCAL ] DIRECTORY [ directory_path ]
Specifies one or more options for the writing of the file format.
+* **row_format**
+
+ Specifies the row format for this insert. Valid options are `SERDE` clause and `DELIMITED` clause. `SERDE` clause can be used to specify a custom `SerDe` for this insert. Alternatively, `DELIMITED` clause can be used to specify the native `SerDe` and state the delimiter, escape character, null character, and so on.
+
+* **hive_serde**
+
+ Specifies the file format for this insert. Valid options are `TEXTFILE`, `SEQUENCEFILE`, `RCFILE`, `ORC`, `PARQUET`, and `AVRO`. You can also specify your own input and output format using `INPUTFORMAT` and `OUTPUTFORMAT`.
+
+* **hive_format**
Review comment:
this should come before `row_format`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943300480
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48731/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943217321
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144251/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943217769
**[Test build #144252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144252/testReport)** for PR 34282 at commit [`3811987`](https://github.com/apache/spark/commit/3811987710c3e8535a7cdf6ef90c042d2a549390).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728791074
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format: `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde`: `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
+
+The inserted rows can be specified by value expressions or result from a query.
### Syntax
```sql
INSERT OVERWRITE [ LOCAL ] DIRECTORY [ directory_path ]
- USING file_format [ OPTIONS ( key = val [ , ... ] ) ]
+ { USING spark_file_format [ OPTIONS ( key = val [ , ... ] ) ] | [ ROW FORMAT row_format ] [ STORED AS hive_file_format ] }
{ VALUES ( { value | NULL } [ , ... ] ) [ , ( ... ) ] | query }
```
### Parameters
* **directory_path**
- Specifies the destination directory. It can also be specified in `OPTIONS` using `path`.
- The `LOCAL` keyword is used to specify that the directory is on the local file system.
+ Specifies the destination directory. The `LOCAL` keyword is used to specify that the directory is on the local file system.
+ In spark file format, it can also be specified in `OPTIONS` using `path`, but directory_path and `path` in OPTIONS should be specified one.
Review comment:
```suggestion
In spark file format, it can also be specified in `OPTIONS` using `path`, but `directory_path` and `path` option can not be both specified.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943300480
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48731/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943117080
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48723/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728673241
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format such as `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde` such as `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
Review comment:
```suggestion
2. Hive format using Hive `Serde`: `ROW FORMAT row_format` or `STORED AS file_format`. Hive support must be enabled to use this command.
```
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format such as `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
Review comment:
```suggestion
1. Spark file format: `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728693479
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format such as `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
Review comment:
DOne
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format such as `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde` such as `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943327055
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48734/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728791074
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -21,31 +21,44 @@ license: |
### Description
-The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using a given Spark file format. The inserted rows can be specified by value expressions or result from a query.
+The `INSERT OVERWRITE DIRECTORY` statement overwrites the existing data in the directory with the new values using following two formats:
+
+ 1. Spark file format: `USING file_format [ OPTIONS ( ket = val [ , ... ] ) ]`.
+ 2. Hive format using Hive `Serde`: `[ ROW FORMAT row_format ] [ STORED AS file_format ]`. Hive support must be enabled to use this command.
+
+The inserted rows can be specified by value expressions or result from a query.
### Syntax
```sql
INSERT OVERWRITE [ LOCAL ] DIRECTORY [ directory_path ]
- USING file_format [ OPTIONS ( key = val [ , ... ] ) ]
+ { USING spark_file_format [ OPTIONS ( key = val [ , ... ] ) ] | [ ROW FORMAT row_format ] [ STORED AS hive_file_format ] }
{ VALUES ( { value | NULL } [ , ... ] ) [ , ( ... ) ] | query }
```
### Parameters
* **directory_path**
- Specifies the destination directory. It can also be specified in `OPTIONS` using `path`.
- The `LOCAL` keyword is used to specify that the directory is on the local file system.
+ Specifies the destination directory. The `LOCAL` keyword is used to specify that the directory is on the local file system.
+ In spark file format, it can also be specified in `OPTIONS` using `path`, but directory_path and `path` in OPTIONS should be specified one.
Review comment:
```suggestion
In spark file format, it can also be specified in `OPTIONS` using `path`, but `directory_path` and `path` options can not be both specified.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943170841
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48723/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34282:
URL: https://github.com/apache/spark/pull/34282#discussion_r728853793
##########
File path: docs/sql-ref-syntax-dml-insert-overwrite-directory.md
##########
@@ -46,6 +57,18 @@ INSERT OVERWRITE [ LOCAL ] DIRECTORY [ directory_path ]
Specifies one or more options for the writing of the file format.
+* **row_format**
+
+ Specifies the row format for this insert. Valid options are `SERDE` clause and `DELIMITED` clause. `SERDE` clause can be used to specify a custom `SerDe` for this insert. Alternatively, `DELIMITED` clause can be used to specify the native `SerDe` and state the delimiter, escape character, null character, and so on.
+
+* **hive_serde**
+
+ Specifies the file format for this insert. Valid options are `TEXTFILE`, `SEQUENCEFILE`, `RCFILE`, `ORC`, `PARQUET`, and `AVRO`. You can also specify your own input and output format using `INPUTFORMAT` and `OUTPUTFORMAT`.
+
+* **hive_format**
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943080137
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48717/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34282: [SPARK-37003][DOC] Merge INSERT related doc into two part: 1. table 2. directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34282:
URL: https://github.com/apache/spark/pull/34282#issuecomment-943269951
**[Test build #144255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144255/testReport)** for PR 34282 at commit [`81971fe`](https://github.com/apache/spark/commit/81971fe77055ca3bb11dc99e86a528f7d370f0b9).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org