You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/05/27 04:16:08 UTC
[GitHub] [incubator-doris] sceneryback opened a new issue #3697: kafka load issue - Unable to load json correctly
sceneryback opened a new issue #3697:
URL: https://github.com/apache/incubator-doris/issues/3697
version: apachedoris/doris-dev:build-env-1.2
Json format in kafka, not array:
```
{
"id": "xxx",
"a": {
"id": "xxx"
}
}
```
a.id is in format like '11:22:33:44:55:66'
Create Table:
```
CREATE TABLE `test` (
`a_id` varchar(24) not null
)
DUPLICATE KEY(`a_id`)
DISTRIBUTED BY HASH(`a_id`) BUCKETS 4
PROPERTIES (
"storage_type" = "COLUMN",
"replication_num" = "1"
);
```
Create load routine:
```
CREATE ROUTINE LOAD testdb.test_load ON test
COLUMNS(a_id)
PROPERTIES
(
"desired_concurrent_number"="3",
"max_batch_interval" = "20",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200",
"strict_mode" = "false",
"format" = "json",
"jsonpaths" = "[\"$.a.id\"]"
)
FROM KAFKA
(
"kafka_broker_list" = "xxx",
"kafka_topic" = "yyy"
);
```
BUT results in table are:
```
select distinct a_id from test limit 5
```
+----------+
| 1 |
| 1 |
| 1 |
| 1 |
| 1 |
+----------+
Seems it only loads the first character in a.id
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #3697: kafka load issue - Unable to load json correctly
Posted by GitBox <gi...@apache.org>.
morningman closed issue #3697:
URL: https://github.com/apache/incubator-doris/issues/3697
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] worker24h commented on issue #3697: kafka load issue - Unable to load json correctly
Posted by GitBox <gi...@apache.org>.
worker24h commented on issue #3697:
URL: https://github.com/apache/incubator-doris/issues/3697#issuecomment-634512507
I fixed the bug.
PR： https://github.com/apache/incubator-doris/pull/3702
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org