You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by mo...@apache.org on 2021/11/09 02:02:38 UTC
[incubator-doris] branch master updated: [Docs] Add hdfs outfile
example (#7052)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 5d946cc [Docs] Add hdfs outfile example (#7052)
5d946cc is described below
commit 5d946ccd5e39f801082c495884f96a93a8bdd490
Author: EmmyMiao87 <52...@qq.com>
AuthorDate: Tue Nov 9 10:02:28 2021 +0800
[Docs] Add hdfs outfile example (#7052)
---
docs/en/administrator-guide/outfile.md | 30 +++++++++++++++------
docs/zh-CN/administrator-guide/outfile.md | 44 +++++++++++++++++++++----------
2 files changed, 52 insertions(+), 22 deletions(-)
diff --git a/docs/en/administrator-guide/outfile.md b/docs/en/administrator-guide/outfile.md
index e2dd8c2..3b5dfe3 100644
--- a/docs/en/administrator-guide/outfile.md
+++ b/docs/en/administrator-guide/outfile.md
@@ -64,7 +64,7 @@ INTO OUTFILE "file_path"
Specify the relevant attributes. Currently it supports exporting through the Broker process, or through the S3, HDFS protocol.
+ Broker related attributes need to be prefixed with `broker.`. For details, please refer to [Broker Document](./broker.html).
- + HDFS protocal can directly execute HDFS protocal configuration.
+ + HDFS protocal can directly execute HDFS protocal configuration. hdfs.fs.defaultFS is used to fill in the namenode address and port. It is required.
+ S3 protocol can directly execute S3 protocol configuration.
```
@@ -141,11 +141,11 @@ Planning example for concurrent export:
1. Example 1
- Export simple query results to the file `hdfs:/path/to/result.txt`. Specify the export format as CSV. Use `my_broker` and set kerberos authentication information. Specify the column separator as `,` and the line delimiter as `\n`.
+ Export simple query results to the file `hdfs://path/to/result.txt`. Specify the export format as CSV. Use `my_broker` and set kerberos authentication information. Specify the column separator as `,` and the line delimiter as `\n`.
```
SELECT * FROM tbl
- INTO OUTFILE "hdfs:/path/to/result_"
+ INTO OUTFILE "hdfs://path/to/result_"
FORMAT AS CSV
PROPERTIES
(
@@ -165,11 +165,11 @@ Planning example for concurrent export:
2. Example 2
- Export simple query results to the file `hdfs:/path/to/result.parquet`. Specify the export format as PARQUET. Use `my_broker` and set kerberos authentication information.
+ Export simple query results to the file `hdfs://path/to/result.parquet`. Specify the export format as PARQUET. Use `my_broker` and set kerberos authentication information.
```
SELECT c1, c2, c3 FROM tbl
- INTO OUTFILE "hdfs:/path/to/result_"
+ INTO OUTFILE "hdfs://path/to/result_"
FORMAT AS PARQUET
PROPERTIES
(
@@ -185,7 +185,7 @@ Planning example for concurrent export:
3. Example 3
- Export the query result of the CTE statement to the file `hdfs:/path/to/result.txt`. The default export format is CSV. Use `my_broker` and set hdfs high availability information. Use the default column separators and line delimiter.
+ Export the query result of the CTE statement to the file `hdfs://path/to/result.txt`. The default export format is CSV. Use `my_broker` and set hdfs high availability information. Use the default column separators and line delimiter.
```
WITH
@@ -194,7 +194,7 @@ Planning example for concurrent export:
x2 AS
(SELECT k3 FROM tbl2)
SELEC k1 FROM x1 UNION SELECT k3 FROM x2
- INTO OUTFILE "hdfs:/path/to/result_"
+ INTO OUTFILE "hdfs://path/to/result_"
PROPERTIES
(
"broker.name" = "my_broker",
@@ -300,7 +300,21 @@ Planning example for concurrent export:
**But because the query statement has a top-level sorting node, even if the query is enabled for concurrently exported session variables, it cannot be exported concurrently.**
-7. Example 7
+8. Example 8
+
+ Use hdfs to export and export the simple query results to the file `hdfs://path/to/result.txt`. Specify the export format as csv.
+
+ ```
+ select * from tbl
+ into outfile "hdfs://path/to/result_"
+ format as csv
+ properties
+ (
+ "hdfs.fs.defaultfs" = "hdfs://namenode:port",
+ );
+ ```
+
+9. Example 9
Export simple query results to the file `hdfs://path/to/result.txt`. Specify the export format as CSV. Use HDFS protocal directly and set kerberos authentication information.
diff --git a/docs/zh-CN/administrator-guide/outfile.md b/docs/zh-CN/administrator-guide/outfile.md
index 762ce21..52b8345 100644
--- a/docs/zh-CN/administrator-guide/outfile.md
+++ b/docs/zh-CN/administrator-guide/outfile.md
@@ -65,7 +65,7 @@ INTO OUTFILE "file_path"
指定相关属性。目前支持通过 Broker 进程, 或通过 S3 协议进行导出。
+ Broker 相关属性需加前缀 `broker.`。具体参阅[Broker 文档](./broker.html)。
- + HDFS 相关属性需加前缀 `hdfs.`。
+ + HDFS 相关属性需加前缀 `hdfs.` 其中 hdfs.fs.defaultFS 用于填写 namenode 地址和端口。属于必填项。。
+ S3 协议则直接执行 S3 协议配置即可。
```
@@ -140,11 +140,11 @@ explain select xxx from xxx where xxx into outfile "s3://xxx" format as csv pro
1. 示例1
- 使用 broker 方式导出，将简单查询结果导出到文件 `hdfs:/path/to/result.txt`。指定导出格式为 CSV。使用 `my_broker` 并设置 kerberos 认证信息。指定列分隔符为 `,`，行分隔符为 `\n`。
+ 使用 broker 方式导出，将简单查询结果导出到文件 `hdfs://path/to/result.txt`。指定导出格式为 CSV。使用 `my_broker` 并设置 kerberos 认证信息。指定列分隔符为 `,`，行分隔符为 `\n`。
```
SELECT * FROM tbl
- INTO OUTFILE "hdfs:/path/to/result_"
+ INTO OUTFILE "hdfs://path/to/result_"
FORMAT AS CSV
PROPERTIES
(
@@ -164,11 +164,11 @@ explain select xxx from xxx where xxx into outfile "s3://xxx" format as csv pro
2. 示例2
- 将简单查询结果导出到文件 `hdfs:/path/to/result.parquet`。指定导出格式为 PARQUET。使用 `my_broker` 并设置 kerberos 认证信息。
+ 将简单查询结果导出到文件 `hdfs://path/to/result.parquet`。指定导出格式为 PARQUET。使用 `my_broker` 并设置 kerberos 认证信息。
```
SELECT c1, c2, c3 FROM tbl
- INTO OUTFILE "hdfs:/path/to/result_"
+ INTO OUTFILE "hdfs://path/to/result_"
FORMAT AS PARQUET
PROPERTIES
(
@@ -184,7 +184,7 @@ explain select xxx from xxx where xxx into outfile "s3://xxx" format as csv pro
3. 示例3
- 将 CTE 语句的查询结果导出到文件 `hdfs:/path/to/result.txt`。默认导出格式为 CSV。使用 `my_broker` 并设置 hdfs 高可用信息。使用默认的行列分隔符。
+ 将 CTE 语句的查询结果导出到文件 `hdfs://path/to/result.txt`。默认导出格式为 CSV。使用 `my_broker` 并设置 hdfs 高可用信息。使用默认的行列分隔符。
```
WITH
@@ -193,7 +193,7 @@ explain select xxx from xxx where xxx into outfile "s3://xxx" format as csv pro
x2 AS
(SELECT k3 FROM tbl2)
SELEC k1 FROM x1 UNION SELECT k3 FROM x2
- INTO OUTFILE "hdfs:/path/to/result_"
+ INTO OUTFILE "hdfs://path/to/result_"
PROPERTIES
(
"broker.name" = "my_broker",
@@ -301,20 +301,36 @@ explain select xxx from xxx where xxx into outfile "s3://xxx" format as csv pro
7. 示例7
- 使用 hdfs 方式导出，将简单查询结果导出到文件 `hdfs:/path/to/result.txt`。指定导出格式为 CSV。使用并设置 kerberos 认证信息。
+ 使用 hdfs 方式导出，将简单查询结果导出到文件 `hdfs://path/to/result.txt`。指定导出格式为 csv。
```
- SELECT * FROM tbl
- INTO OUTFILE "hdfs://path/to/result_"
- FORMAT AS CSV
- PROPERTIES
+ select * from tbl
+ into outfile "hdfs://path/to/result_"
+ format as csv
+ properties
+ (
+ "hdfs.fs.defaultfs" = "hdfs://namenode:port",
+ );
+ ```
+
+
+8. 示例8
+
+ 使用 hdfs 方式导出，将简单查询结果导出到文件 `hdfs://path/to/result.txt`。指定导出格式为 csv。使用并设置 kerberos 认证信息。
+
+ ```
+ select * from tbl
+ into outfile "hdfs://path/to/result_"
+ format as csv
+ properties
(
- "hdfs.fs.defaultFS" = "hdfs://namenode:port",
+ "hdfs.fs.defaultfs" = "hdfs://namenode:port",
"hdfs.hadoop.security.authentication" = "kerberos",
- "hdfs.kerberos_principal" = "doris@YOUR.COM",
+ "hdfs.kerberos_principal" = "doris@your.com",
"hdfs.kerberos_keytab" = "/home/doris/my.keytab"
);
```
+
## 返回结果
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org