You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/12/06 07:24:46 UTC
[GitHub] [incubator-doris] qidaye opened a new issue #7322: [Bug] Can not use stream as identifier
qidaye opened a new issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Version
0.14.13 and above
### What's Wrong?
In 0.13 and previous versions, user can use `stream` as an identifier.
For example:
```sql
CREATE ROUTINE LOAD cdn_stream_stats_routine_load_v2 ON cdn_stream_stats_v2
COLUMNS TERMINATED BY "," ,
COLUMNS(domain,host,conn,stream,ts,node,peer,score,videofps,videorate,flowrate,tcprtt,videofpsvar)
PROPERTIES
(
"desired_concurrent_number"="4",
"max_batch_interval" = "20",
"max_batch_rows" = "6000000",
"max_batch_size" = "209715200",
"max_error_number"= "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "xxx",
"kafka_topic" = "xx",
"property.security.protocol" = "ssl",
"property.ssl.ca.location" = "FILE:ca.pem",
"property.ssl.certificate.location" = "FILE:client.pem",
"property.ssl.key.location" = "FILE:client.key",
"property.client.id" = "xxx"
)
```
But when upgrade Doris to 0.14.13, the sql is parsing error after restarted FE.
```
java.io.IOException: error happens when parsing create routine load stmt:
```
When execute a sql contain `stream` as identifier, it occurs error too.
```
ERROR 1105 (HY000): errCode = 2, detailMessage = Syntax error in line 3:
COLUMNS(domain,host,conn,stream,ts,node,peer,score,vi...
^
Encountered: STREAM
Expected: STREAM is keyword, maybe `STREAM`
```
### What You Expected?
Execute normally.
### How to Reproduce?
# upgrade
1. Set up a Doris cluster in 0.13.16.
2. Create table
```sql
CREATE TABLE `cdn_stream_stats_v2` (
`domain` varchar(128) NOT NULL DEFAULT "" COMMENT "",
`host` varchar(32) NOT NULL DEFAULT "" COMMENT "",
`conn` varchar(32) NOT NULL DEFAULT "" COMMENT "",
`stream` varchar(128) NOT NULL DEFAULT "" COMMENT "",
`ts` datetime NOT NULL COMMENT "",
`node` varchar(32) REPLACE NOT NULL DEFAULT "" COMMENT "",
`peer` varchar(128) REPLACE NOT NULL DEFAULT "" COMMENT "",
`score` int(11) REPLACE NOT NULL DEFAULT "0" COMMENT "",
`videofps` float REPLACE NOT NULL DEFAULT "0" COMMENT "",
`videorate` int(11) REPLACE NOT NULL DEFAULT "0" COMMENT "",
`flowrate` int(11) REPLACE NOT NULL DEFAULT "0" COMMENT "",
`tcprtt` int(11) REPLACE NOT NULL DEFAULT "0" COMMENT "",
`videofpsvar` float REPLACE NOT NULL DEFAULT "0" COMMENT ""
) ENGINE=OLAP
AGGREGATE KEY(`domain`, `host`, `conn`, `stream`, `ts`)
COMMENT "OLAP"
PARTITION BY RANGE(`ts`)
(PARTITION pw2021_17 VALUES [('2021-04-27 00:00:00'), ('2021-04-29 00:00:00')),
PARTITION pw2021_18 VALUES [('2021-04-29 00:00:00'), ('2021-05-06 00:00:00')),
PARTITION pw2021_19 VALUES [('2021-05-06 00:00:00'), ('2021-05-13 00:00:00')),
PARTITION pw2021_20 VALUES [('2021-05-13 00:00:00'), ('2021-05-20 00:00:00')))
DISTRIBUTED BY HASH(`stream`) BUCKETS 1
PROPERTIES (
"replication_num" = "1",
"dynamic_partition.enable" = "false",
"dynamic_partition.time_unit" = "WEEK",
"dynamic_partition.time_zone" = "Asia/Shanghai",
"dynamic_partition.start" = "-2",
"dynamic_partition.end" = "2",
"dynamic_partition.prefix" = "pw",
"dynamic_partition.replication_num" = "2",
"dynamic_partition.buckets" = "16",
"dynamic_partition.start_day_of_week" = "4",
"in_memory" = "false",
"storage_format" = "V2"
);
```
3. Create routine load
```sql
CREATE ROUTINE LOAD cdn_stream_stats_routine_load_v2 ON cdn_stream_stats_v2
COLUMNS TERMINATED BY "," ,
COLUMNS(domain,host,conn,stream,ts,node,peer,score,videofps,videorate,flowrate,tcprtt,videofpsvar)
PROPERTIES
(
"desired_concurrent_number"="4",
"max_batch_interval" = "20",
"max_batch_rows" = "6000000",
"max_batch_size" = "209715200",
"max_error_number"= "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "xxx",
"kafka_topic" = "xxx",
"property.security.protocol" = "ssl",
"property.ssl.ca.location" = "FILE:ca.pem",
"property.ssl.certificate.location" = "FILE:client.pem",
"property.ssl.key.location" = "FILE:client.key",
"property.client.id" = "xxx"
);
```
4. Upgrade Doris to 0.14.13
5. The FE won't start up.
# Current
1. Set up a current version of Doris
2. Create table as above.
3. Create routine load as above.
4. Occurs error.
### Anything Else?
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WindyGao commented on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
WindyGao commented on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-1004546478
Which field might be wrong?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WindyGao commented on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
WindyGao commented on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-995756830
> > Is there some others keyword may exsist like 'stream'?
>
> You can apply this patch to show the orgin routine load stmt in `fe.out`.
>
> Is there any `stream` column in your routine load destination table?
I can only confirm that 'stream' does not exist in the columns of all tables by select information_schema
But I cannot confirm whether 'stream' exists in all routine load Job...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
morningman closed issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
morningman commented on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-1004547985
you just copy and execute that routine load stmt in mysql cli to see what's wrong
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] qidaye commented on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
qidaye commented on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-995785812
@WindyGao You can apply this path to print the origin load stmt, then you can confirm which keyword is wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WindyGao commented on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
WindyGao commented on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-995798536
> @WindyGao You can apply this path to print the origin load stmt, then you can confirm which keyword is wrong.
you mean 'origStmt.originStmt'?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WindyGao commented on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
WindyGao commented on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-1004547319
type,struct,name
parent
There are several possibilities...
i need to verfiy it by upgrading fe version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WindyGao edited a comment on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
WindyGao edited a comment on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-1004546478
Which field might be wrong?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] qidaye commented on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
qidaye commented on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-996377885
>
Yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] qidaye commented on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
qidaye commented on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-995723844
> Is there some others keyword may exsist like 'stream'?
You can apply this patch to show the orgin routine load stmt in `fe.out`.
Is there any `stream` column in your routine load destination table?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WindyGao edited a comment on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
WindyGao edited a comment on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-995706183
Is there some others keyword may exsist like 'stream'?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WindyGao commented on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
WindyGao commented on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-995706183
Is there some others keyword like 'stream'?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WindyGao commented on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
WindyGao commented on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-1004552512
> you just copy and execute that routine load stmt in mysql cli to see what's wrong
...good idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WindyGao commented on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
WindyGao commented on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-995705767
I also encountered this problem ,but I don't find any 'stream' column which exsists in my dev cluster
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] qidaye edited a comment on issue #7322: [Bug] Can not use stream as identifier
Posted by GitBox <gi...@apache.org>.
qidaye edited a comment on issue #7322:
URL: https://github.com/apache/incubator-doris/issues/7322#issuecomment-996377885
@WindyGao yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org