[GitHub] [incubator-doris] imay commented on a change in pull request #1569: Enable Partition Discovery for Broker Load

[GitBox <gi...@apache.org>]

imay commented on a change in pull request #1569: Enable Partition Discovery for Broker Load
URL: https://github.com/apache/incubator-doris/pull/1569#discussion_r310419097
 
 

 ##########
 File path: docs/help/Contents/Data Manipulation/broker_load.md
 ##########
 @@ -352,15 +357,47 @@
         )
         WITH BROKER hdfs ("username"="hdfs_user", "password"="hdfs_password");
 
-     8. 导入Parquet文件中数据  指定FORMAT 为parquet， 默认是通过文件后缀判断
+    8. 导入Parquet文件中数据  指定FORMAT 为parquet， 默认是通过文件后缀判断
         LOAD LABEL example_db.label9
         (
         DATA INFILE("hdfs://hdfs_host:hdfs_port/user/palo/data/input/file")
         INTO TABLE `my_table`
         FORMAT AS "parquet"
         (k1, k2, k3)
         )
-        WITH BROKER hdfs ("username"="hdfs_user", "password"="hdfs_password");
+        WITH BROKER hdfs ("username"="hdfs_user", "password"="hdfs_password"); 
+
+    9. 通过Partition Discovery提取文件路径中的压缩字段
+        如果导入路径为目录，则递归地列出该目录下的所有parquet文件
+        如果需要，则会根据表中定义的字段类型解析文件路径中的partitioned fields，实现类似Spark中读parquet文件
+        1. 不指定Partition Discovery的基础路径（BASE_PATH）
+            LOAD LABEL example_db.label10
+            (
+            DATA INFILE("hdfs://hdfs_host:hdfs_port/user/palo/data/input/dir")
 
 Review comment:
   这样直接指定目录就可以导入，跟我们之前的默认行为是有冲突的。有的时候用户写错了，可能会将整个目录导入。
   为了避免这种情况，可以不使用DATA INFILE，可是使用类似DATA INDIR的方式来指定导入一个目录

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@doris.apache.org
For additional commands, e-mail: dev-help@doris.apache.org