You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/03/08 13:09:04 UTC
[GitHub] [incubator-doris] francisoliverlee opened a new issue #5486: [Bug]Broker export Duplicate data to hdfs
francisoliverlee opened a new issue #5486:
URL: https://github.com/apache/incubator-doris/issues/5486
**Describe the bug**
Broker export Duplicate data to hdfs
**To Reproduce**
- create table sql
```sql
CREATE TABLE `vk_dwd_9m_kpi_new_da` (
`xxxxxxx` varchar(50) NULL COMMENT "",
`type` varchar(50) NULL COMMENT " ",
`kpi` varchar(50) NULL COMMENT "",
`channel1` varchar(50) NULL COMMENT "",
`channel2` varchar(50) NULL COMMENT "",
`target` decimal(18, 4) NULL COMMENT " ",
`mtd_target` decimal(18, 4) NULL COMMENT " "
) ENGINE=OLAP
DUPLICATE KEY(`xxxxxxx`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`xxxxxxx`) BUCKETS 3
PROPERTIES (
"replication_num" = "3",
"in_memory" = "false",
"storage_format" = "DEFAULT"
);
```
- export sql
```sql
EXPORT TABLE vk_dw.vk_dwd_9m_kpi_new_da TO "hdfs://vipkidcold/user/lixinshang/"
PROPERTIES
(
"exec_mem_limit"="2147483648",
"timeout" = "3600",
"column_separator"="$"
)
WITH BROKER "hdfs_broker"
(
"hadoop.security.authentication" = "simple",
"username" = "xxxx",
"password" = "xxxxxx+CS",
"dfs.nameservices" = "vipkidcold",
"dfs.ha.namenodes.vipkidcold" = "nn1,nn2",
"dfs.namenode.rpc-address.vipkidcold.nn1" = "l-xxxxxx:8020",
"dfs.namenode.rpc-address.vipkidcold.nn2" = "l-xxxxx2:8020"
```
- wrong result on hdfs: checked, same rows in those hdfs files
![Uploading image.png…]()
- right result on hdfs(use another account to try)
**Expected behavior**
**Smartphone (please complete the following information):**
- Version : 0.12.21
i doublt : exec plan wrong.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] francisoliverlee commented on issue #5486: [Bug]Broker export Duplicate data to hdfs
Posted by GitBox <gi...@apache.org>.
francisoliverlee commented on issue #5486:
URL: https://github.com/apache/incubator-doris/issues/5486#issuecomment-793458393
checked.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] francisoliverlee closed issue #5486: [Bug]Broker export Duplicate data to hdfs
Posted by GitBox <gi...@apache.org>.
francisoliverlee closed issue #5486:
URL: https://github.com/apache/incubator-doris/issues/5486
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org