You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "sohardforaname (via GitHub)" <gi...@apache.org> on 2023/07/25 08:16:31 UTC
[GitHub] [doris] sohardforaname opened a new pull request, #22195: [Feature](Nereids)support create table and CTAS for Nereids
sohardforaname opened a new pull request, #22195:
URL: https://github.com/apache/doris/pull/22195
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1657843954
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1661904755
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1661965333
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.15 seconds
stream load tsv: 497 seconds loaded 74807831229 Bytes, about 143 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 25.2 seconds inserted 10000000 Rows, about 396K ops/s
storage size: 17072070649 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morningman commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1707884432
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712759593
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1659510813
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.14 seconds
stream load tsv: 496 seconds loaded 74807831229 Bytes, about 143 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 25.1 seconds inserted 10000000 Rows, about 398K ops/s
storage size: 17072070692 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1657216476
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1664078578
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1687734406
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.51 seconds
stream load tsv: 523 seconds loaded 74807831229 Bytes, about 136 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 61 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 25.7 seconds inserted 10000000 Rows, about 389K ops/s
storage size: 17072071000 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712537591
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.79 seconds
stream load tsv: 549 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17161832602 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1659488551
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1658429918
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1695687782
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 50.31 seconds
stream load tsv: 523 seconds loaded 74807831229 Bytes, about 136 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 25.3 seconds inserted 10000000 Rows, about 395K ops/s
storage size: 17071999348 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #22195: [Feature](Nereids)framework of create table and CTAS for Nereids.
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #22195:
URL: https://github.com/apache/doris/pull/22195#discussion_r1315335263
##########
regression-test/suites/nereids_p0/create_table/test_create_table.groovy:
##########
Review Comment:
add complex test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1711249353
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712850016
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1687683063
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1709749542
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1711300997
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.05 seconds
stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162158602 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #22195:
URL: https://github.com/apache/doris/pull/22195#discussion_r1315336684
##########
regression-test/suites/nereids_p0/create_table/test_ctas.groovy:
##########
Review Comment:
add nullable and date->datev2, datetime->datetimev2, decimalv2->decimalv3 test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1661962910
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.87 seconds
stream load tsv: 498 seconds loaded 74807831229 Bytes, about 143 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 25.1 seconds inserted 10000000 Rows, about 398K ops/s
storage size: 17072070608 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1661699963
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow closed pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow closed pull request #22195: [Feature](Nereids) support create table and CTAS
URL: https://github.com/apache/doris/pull/22195
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712732122
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712851508
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1689718515
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.52 seconds
stream load tsv: 522 seconds loaded 74807831229 Bytes, about 136 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 61 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 25.8 seconds inserted 10000000 Rows, about 387K ops/s
storage size: 17072002728 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] selectdb-robot commented on pull request #22195: [Feature](Nereids)framework of create table and CTAS for Nereids.
Posted by "selectdb-robot (via GitHub)" <gi...@apache.org>.
selectdb-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1705458922
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.21 seconds
stream load tsv: 535 seconds loaded 74807831229 Bytes, about 133 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.3 seconds inserted 10000000 Rows, about 341K ops/s
storage size: 17162060374 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1680210150
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)framework of create table and CTAS for Nereids.
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1705076300
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1711368827
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1708020604
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.18 seconds
stream load tsv: 534 seconds loaded 74807831229 Bytes, about 133 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17162100295 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1689388296
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1689512140
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1711426514
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.63 seconds
stream load tsv: 545 seconds loaded 74807831229 Bytes, about 130 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17162117601 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1706610186
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] selectdb-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "selectdb-robot (via GitHub)" <gi...@apache.org>.
selectdb-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1706664354
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.18 seconds
stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162272401 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712870095
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.91 seconds
stream load tsv: 600 seconds loaded 74807831229 Bytes, about 118 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162470492 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1680356288
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.16 seconds
stream load tsv: 521 seconds loaded 74807831229 Bytes, about 136 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 95.8 seconds inserted 10000000 Rows, about 104K ops/s
storage size: 17072071220 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1689467514
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712557905
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712824675
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.17 seconds
stream load tsv: 584 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17162438261 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712867472
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.83 seconds
stream load tsv: 595 seconds loaded 74807831229 Bytes, about 119 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17162320174 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712704378
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #22195: [Feature](Nereids)framework of create table and CTAS for Nereids.
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #22195:
URL: https://github.com/apache/doris/pull/22195#discussion_r1315305569
##########
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisLexer.g4:
##########
@@ -177,16 +178,16 @@ DATA: 'DATA';
DATABASE: 'DATABASE';
DATABASES: 'DATABASES';
DATE: 'DATE';
-DATE_ADD: 'DATE_ADD';
-DATE_CEIL: 'DATE_CEIL';
-DATE_DIFF: 'DATE_DIFF';
-DATE_FLOOR: 'DATE_FLOOR';
-DATE_SUB: 'DATE_SUB';
DATEADD: 'DATEADD';
DATEDIFF: 'DATEDIFF';
DATETIME: 'DATETIME';
DATETIMEV2: 'DATETIMEV2';
DATEV2: 'DATEV2';
+DATE_ADD: 'DATE_ADD';
Review Comment:
remove unnecessary changes
##########
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4:
##########
@@ -315,6 +325,72 @@ tableAlias
multipartIdentifier
: parts+=errorCapturingIdentifier (DOT parts+=errorCapturingIdentifier)*
;
+
+// ----------------Create Table Fields----------
+
+columnDefs
+ : cols+=columnDef (COMMA cols+=columnDef)*
+ ;
+
+columnDef
+ : colName=identifier type=dataType
+ KEY? (aggType=aggTypeDef)? ((NOT NULL) | NULL)?
+ (DEFAULT (defaultValue=constant | CURRENT_TIMESTAMP))?
+ (COMMENT comment=constant)?
+ ;
+
+indexDefs
+ : indexes+=indexDef (COMMA indexes+=indexDef)*
+ ;
+
+indexDef
+ : INDEX indexName=identifier cols=identifierList (USING BITMAP)? (comment=constant)?
Review Comment:
comment should not be `constant`
##########
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4:
##########
@@ -40,6 +40,16 @@ statement
AS type=(RESTRICTIVE | PERMISSIVE)
TO (user=userIdentify | ROLE roleName=identifier)
USING LEFT_PAREN booleanExpression RIGHT_PAREN #createRowPolicy
+ | CREATE TABLE (IF NOT EXISTS)? name=multipartIdentifier
+ ((LEFT_PAREN columnDefs indexDefs? RIGHT_PAREN) | (ctasCols=identifierList)?)
+ (ENGINE EQ engine=identifier)?
+ ((AGGREGATE | UNIQUE | DUPLICATE) KEY keys=identifierList)?
+ (COMMENT constant)?
Review Comment:
comment should not `constant`
##########
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4:
##########
@@ -315,6 +325,72 @@ tableAlias
multipartIdentifier
: parts+=errorCapturingIdentifier (DOT parts+=errorCapturingIdentifier)*
;
+
+// ----------------Create Table Fields----------
+
+columnDefs
+ : cols+=columnDef (COMMA cols+=columnDef)*
+ ;
+
+columnDef
+ : colName=identifier type=dataType
+ KEY? (aggType=aggTypeDef)? ((NOT NULL) | NULL)?
+ (DEFAULT (defaultValue=constant | CURRENT_TIMESTAMP))?
+ (COMMENT comment=constant)?
+ ;
+
+indexDefs
+ : indexes+=indexDef (COMMA indexes+=indexDef)*
+ ;
+
+indexDef
+ : INDEX indexName=identifier cols=identifierList (USING BITMAP)? (comment=constant)?
+ ;
+
+partitionsDef
+ : partitions+=partitionDef (COMMA partitions+=partitionDef)*
+ ;
+
+partitionDef
+ : lessThanPartitionDef
+ | fixedPartitionDef
+ | stepPartitionDef
+ | inPartitionDef
+ ;
+
+lessThanPartitionDef
+ : PARTITION partitionName=identifier VALUES LESS THAN (MAXVALUE | constantSeq)
Review Comment:
miss `if not exists` in partitionDef
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -1718,6 +1745,194 @@ private LogicalPlan plan(ParserRuleContext tree) {
return (LogicalPlan) tree.accept(this);
}
+ /* ********************************************************************************************
+ * create table parsing
+ * ******************************************************************************************** */
+
+ @Override
+ public LogicalPlan visitCreateTable(CreateTableContext ctx) {
+ String dbName = null;
+ String tableName;
+ List<String> nameParts = visitMultipartIdentifier(ctx.name);
+ if (nameParts.size() == 1) {
+ tableName = nameParts.get(0);
+ } else if (nameParts.size() == 2) {
+ dbName = nameParts.get(0);
+ tableName = nameParts.get(1);
+ } else {
+ throw new AnalysisException("nameParts in create table should be 1 or 2");
+ }
+ KeysType keysType = null;
+ if (ctx.DUPLICATE() != null) {
+ keysType = KeysType.DUP_KEYS;
+ } else if (ctx.AGGREGATE() != null) {
+ keysType = KeysType.AGG_KEYS;
+ } else if (ctx.UNIQUE() != null) {
+ keysType = KeysType.UNIQUE_KEYS;
+ }
+ String engineName = ctx.engine != null ? ctx.engine.getText().toLowerCase() : "olap";
+ DistributionDescriptor desc = new DistributionDescriptor(ctx.HASH() != null, ctx.AUTO() != null,
+ Integer.parseInt(ctx.number().getText()),
Review Comment:
if AUTO bucket, i think ctx.number() will return null, and this statement will throw NPE.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -1718,6 +1745,194 @@ private LogicalPlan plan(ParserRuleContext tree) {
return (LogicalPlan) tree.accept(this);
}
+ /* ********************************************************************************************
+ * create table parsing
+ * ******************************************************************************************** */
+
+ @Override
+ public LogicalPlan visitCreateTable(CreateTableContext ctx) {
+ String dbName = null;
+ String tableName;
+ List<String> nameParts = visitMultipartIdentifier(ctx.name);
+ if (nameParts.size() == 1) {
+ tableName = nameParts.get(0);
+ } else if (nameParts.size() == 2) {
+ dbName = nameParts.get(0);
+ tableName = nameParts.get(1);
+ } else {
+ throw new AnalysisException("nameParts in create table should be 1 or 2");
+ }
+ KeysType keysType = null;
+ if (ctx.DUPLICATE() != null) {
+ keysType = KeysType.DUP_KEYS;
+ } else if (ctx.AGGREGATE() != null) {
+ keysType = KeysType.AGG_KEYS;
+ } else if (ctx.UNIQUE() != null) {
+ keysType = KeysType.UNIQUE_KEYS;
+ }
+ String engineName = ctx.engine != null ? ctx.engine.getText().toLowerCase() : "olap";
+ DistributionDescriptor desc = new DistributionDescriptor(ctx.HASH() != null, ctx.AUTO() != null,
+ Integer.parseInt(ctx.number().getText()),
+ ctx.HASH() != null ? visitIdentifierList(ctx.hashKeys) : null);
+ Map<String, String> properties = ctx.propertyClause() != null
+ ? visitPropertyClause(ctx.propertyClause()) : null;
+ String partitionType = null;
+ if (ctx.PARTITION() != null) {
+ partitionType = ctx.RANGE() != null ? "RANGE" : "LIST";
+ }
+
+ if (ctx.columnDefs() != null) {
+ if (ctx.AS() != null) {
+ throw new AnalysisException("Should not define the entire column in CTAS");
+ }
+ return new CreateTableCommand(Optional.empty(), new CreateTableInfo(
+ ctx.EXISTS() != null,
+ dbName,
+ tableName,
+ visitColumnDefs(ctx.columnDefs()),
+ ImmutableList.of(),
+ engineName,
+ keysType,
+ ctx.keys != null ? visitIdentifierList(ctx.keys) : ImmutableList.of(),
+ "",
+ partitionType,
+ ctx.partitionKeys != null ? visitIdentifierList(ctx.partitionKeys) : null,
+ ctx.partitions != null ? visitPartitionsDef(ctx.partitions) : null,
+ desc,
+ ImmutableList.of(),
+ properties));
+ } else if (ctx.AS() != null) {
+ return new CreateTableCommand(Optional.of(visitQuery(ctx.query())), new CreateTableInfo(
+ ctx.EXISTS() != null,
+ dbName,
+ tableName,
+ ctx.ctasCols != null ? visitIdentifierList(ctx.ctasCols) : null,
+ engineName,
+ keysType,
+ ctx.keys != null ? visitIdentifierList(ctx.keys) : ImmutableList.of(),
+ "",
+ partitionType,
+ ctx.partitionKeys != null ? visitIdentifierList(ctx.partitionKeys) : null,
+ ctx.partitions != null ? visitPartitionsDef(ctx.partitions) : null,
+ desc,
+ ImmutableList.of(),
+ properties));
+ } else {
+ throw new AnalysisException("Should contain at least one column in a table");
+ }
+ }
+
+ @Override
+ public List<ColumnDefinition> visitColumnDefs(ColumnDefsContext ctx) {
+ return ctx.cols.stream().map(this::visitColumnDef).collect(Collectors.toList());
Review Comment:
use ImmutableList
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateTableCommand.java:
##########
@@ -0,0 +1,134 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.commands;
+
+import org.apache.doris.analysis.CreateTableStmt;
+import org.apache.doris.analysis.DropTableStmt;
+import org.apache.doris.analysis.TableName;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.nereids.NereidsPlanner;
+import org.apache.doris.nereids.analyzer.UnboundOlapTableSink;
+import org.apache.doris.nereids.annotation.Developing;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.properties.PhysicalProperties;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.PlanType;
+import org.apache.doris.nereids.trees.plans.commands.ExplainCommand.ExplainLevel;
+import org.apache.doris.nereids.trees.plans.commands.info.ColumnDefinition;
+import org.apache.doris.nereids.trees.plans.commands.info.CreateTableInfo;
+import org.apache.doris.nereids.trees.plans.logical.LogicalPlan;
+import org.apache.doris.nereids.trees.plans.visitor.PlanVisitor;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.QueryState.MysqlStateType;
+import org.apache.doris.qe.StmtExecutor;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+/**
+ * create table command
+ */
+@Developing
+public class CreateTableCommand extends Command implements ForwardWithSync {
+ public static final Logger LOG = LogManager.getLogger(CreateTableCommand.class);
+
+ private final Optional<LogicalPlan> ctasQuery;
+ private final CreateTableInfo createTableInfo;
+
+ public CreateTableCommand(Optional<LogicalPlan> ctasQuery, CreateTableInfo createTableInfo) {
+ super(PlanType.CREATE_TABLE_COMMAND);
+ this.ctasQuery = ctasQuery;
+ this.createTableInfo = Objects.requireNonNull(createTableInfo);
+ }
+
+ @Override
+ public void run(ConnectContext ctx, StmtExecutor executor) throws Exception {
+ if (!ctasQuery.isPresent()) {
Review Comment:
i think create table and create table as select should use two different command
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateTableCommand.java:
##########
@@ -0,0 +1,134 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.commands;
+
+import org.apache.doris.analysis.CreateTableStmt;
+import org.apache.doris.analysis.DropTableStmt;
+import org.apache.doris.analysis.TableName;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.nereids.NereidsPlanner;
+import org.apache.doris.nereids.analyzer.UnboundOlapTableSink;
+import org.apache.doris.nereids.annotation.Developing;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.properties.PhysicalProperties;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.PlanType;
+import org.apache.doris.nereids.trees.plans.commands.ExplainCommand.ExplainLevel;
+import org.apache.doris.nereids.trees.plans.commands.info.ColumnDefinition;
+import org.apache.doris.nereids.trees.plans.commands.info.CreateTableInfo;
+import org.apache.doris.nereids.trees.plans.logical.LogicalPlan;
+import org.apache.doris.nereids.trees.plans.visitor.PlanVisitor;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.QueryState.MysqlStateType;
+import org.apache.doris.qe.StmtExecutor;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+/**
+ * create table command
+ */
+@Developing
+public class CreateTableCommand extends Command implements ForwardWithSync {
+ public static final Logger LOG = LogManager.getLogger(CreateTableCommand.class);
+
+ private final Optional<LogicalPlan> ctasQuery;
+ private final CreateTableInfo createTableInfo;
+
+ public CreateTableCommand(Optional<LogicalPlan> ctasQuery, CreateTableInfo createTableInfo) {
+ super(PlanType.CREATE_TABLE_COMMAND);
+ this.ctasQuery = ctasQuery;
+ this.createTableInfo = Objects.requireNonNull(createTableInfo);
+ }
+
+ @Override
+ public void run(ConnectContext ctx, StmtExecutor executor) throws Exception {
+ if (!ctasQuery.isPresent()) {
+ createTableInfo.validate(ctx);
+ CreateTableStmt createTableStmt = createTableInfo.translateToCatalogStyle();
+ LOG.info("Nereids start to execute the create table command, query id: {}, tableName: {}",
+ ctx.queryId(), createTableInfo.getTableName());
+ try {
+ Env.getCurrentEnv().createTable(createTableStmt);
+ } catch (Exception e) {
+ throw new AnalysisException(e.getMessage(), e.getCause());
+ }
+ return;
+ }
+ LogicalPlan query = ctasQuery.get();
+ List<String> ctasCols = createTableInfo.getCtasColumns();
+ NereidsPlanner planner = new NereidsPlanner(ctx.getStatementContext());
+ Plan plan = planner.plan(query, PhysicalProperties.ANY, ExplainLevel.NONE);
+ if (ctasCols == null) {
+ // we should analyze the plan firstly to get the columns' name.
+ ctasCols = plan.getOutput().stream().map(NamedExpression::getName).collect(Collectors.toList());
+ }
+ List<Slot> slots = plan.getOutput();
+ if (slots.size() != ctasCols.size()) {
+ throw new AnalysisException("ctas column size is not equal to the query's");
+ }
+ List<ColumnDefinition> columnsOfQuery = slots.stream()
+ .map(s -> new ColumnDefinition(s.getName(), s.getDataType(), s.nullable()))
+ .collect(Collectors.toList());
Review Comment:
ImmutableList
##########
regression-test/suites/nereids_p0/create_table/test_fnf.groovy:
##########
@@ -0,0 +1,86 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("nereids_test_fnf") {
Review Comment:
what's mean about fnf?
##########
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4:
##########
@@ -315,6 +325,72 @@ tableAlias
multipartIdentifier
: parts+=errorCapturingIdentifier (DOT parts+=errorCapturingIdentifier)*
;
+
+// ----------------Create Table Fields----------
+
+columnDefs
+ : cols+=columnDef (COMMA cols+=columnDef)*
+ ;
+
+columnDef
+ : colName=identifier type=dataType
+ KEY? (aggType=aggTypeDef)? ((NOT NULL) | NULL)?
+ (DEFAULT (defaultValue=constant | CURRENT_TIMESTAMP))?
+ (COMMENT comment=constant)?
+ ;
+
+indexDefs
+ : indexes+=indexDef (COMMA indexes+=indexDef)*
+ ;
+
+indexDef
+ : INDEX indexName=identifier cols=identifierList (USING BITMAP)? (comment=constant)?
+ ;
+
+partitionsDef
+ : partitions+=partitionDef (COMMA partitions+=partitionDef)*
+ ;
+
+partitionDef
+ : lessThanPartitionDef
+ | fixedPartitionDef
+ | stepPartitionDef
+ | inPartitionDef
+ ;
+
+lessThanPartitionDef
+ : PARTITION partitionName=identifier VALUES LESS THAN (MAXVALUE | constantSeq)
+ ;
+
+fixedPartitionDef
+ : PARTITION partitionName=identifier VALUES LEFT_BRACKET lower=constantSeq COMMA upper=constantSeq RIGHT_PAREN
+ ;
+
+stepPartitionDef
+ : FROM from=constantSeq TO to=constantSeq
+ ((INTERVAL unitsAmount=valueExpression unit=datetimeUnit) | unitsAmount=valueExpression)
+ ;
+
+inPartitionDef
+ : PARTITION partitionName=identifier VALUES IN ((LEFT_PAREN constantSeqs+=constantSeq
+ (COMMA constantSeqs+=constantSeq)* RIGHT_PAREN) | constants=constantSeq)
+ ;
+
+constantSeq
+ : LEFT_PAREN values+=constant (COMMA values+=constant)* RIGHT_PAREN
Review Comment:
values should use `INTEGER_LITERAL | STRING_LITERAL | MAX_VALUE`
##########
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4:
##########
@@ -748,6 +824,12 @@ nonReserved
| JOBS
| JSON
| JSONB
+ | INSERT
+ | INTERVAL
+ | INTO
+ | IS
+ | KEY
+ | KEYS
Review Comment:
remove these, these is not non-reserved keywords
##########
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4:
##########
@@ -315,6 +325,72 @@ tableAlias
multipartIdentifier
: parts+=errorCapturingIdentifier (DOT parts+=errorCapturingIdentifier)*
;
+
+// ----------------Create Table Fields----------
+
+columnDefs
+ : cols+=columnDef (COMMA cols+=columnDef)*
+ ;
+
+columnDef
+ : colName=identifier type=dataType
+ KEY? (aggType=aggTypeDef)? ((NOT NULL) | NULL)?
+ (DEFAULT (defaultValue=constant | CURRENT_TIMESTAMP))?
+ (COMMENT comment=constant)?
+ ;
+
+indexDefs
+ : indexes+=indexDef (COMMA indexes+=indexDef)*
+ ;
+
+indexDef
+ : INDEX indexName=identifier cols=identifierList (USING BITMAP)? (comment=constant)?
+ ;
+
+partitionsDef
+ : partitions+=partitionDef (COMMA partitions+=partitionDef)*
+ ;
+
+partitionDef
+ : lessThanPartitionDef
+ | fixedPartitionDef
+ | stepPartitionDef
+ | inPartitionDef
+ ;
+
+lessThanPartitionDef
+ : PARTITION partitionName=identifier VALUES LESS THAN (MAXVALUE | constantSeq)
+ ;
+
+fixedPartitionDef
+ : PARTITION partitionName=identifier VALUES LEFT_BRACKET lower=constantSeq COMMA upper=constantSeq RIGHT_PAREN
+ ;
+
+stepPartitionDef
+ : FROM from=constantSeq TO to=constantSeq
+ ((INTERVAL unitsAmount=valueExpression unit=datetimeUnit) | unitsAmount=valueExpression)
+ ;
+
+inPartitionDef
+ : PARTITION partitionName=identifier VALUES IN ((LEFT_PAREN constantSeqs+=constantSeq
+ (COMMA constantSeqs+=constantSeq)* RIGHT_PAREN) | constants=constantSeq)
+ ;
+
+constantSeq
+ : LEFT_PAREN values+=constant (COMMA values+=constant)* RIGHT_PAREN
+ ;
+
+rollupDefs
+ : rollups+=rollupDef (COMMA rollups+=rollupDef)*
+ ;
+
+rollupDef
+ : rollupName=identifier rollupCols=identifierList
+ ;
+
+aggTypeDef
+ : MAX | MIN | SUM | REPLACE | REPLACE_IF_NOT_NULL | HLL_UNION | BITMAP_UNION
Review Comment:
miss `QUANTILE_UNION`
##########
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4:
##########
@@ -315,6 +325,72 @@ tableAlias
multipartIdentifier
: parts+=errorCapturingIdentifier (DOT parts+=errorCapturingIdentifier)*
;
+
+// ----------------Create Table Fields----------
+
+columnDefs
+ : cols+=columnDef (COMMA cols+=columnDef)*
+ ;
+
+columnDef
+ : colName=identifier type=dataType
+ KEY? (aggType=aggTypeDef)? ((NOT NULL) | NULL)?
+ (DEFAULT (defaultValue=constant | CURRENT_TIMESTAMP))?
+ (COMMENT comment=constant)?
+ ;
+
+indexDefs
+ : indexes+=indexDef (COMMA indexes+=indexDef)*
+ ;
+
+indexDef
+ : INDEX indexName=identifier cols=identifierList (USING BITMAP)? (comment=constant)?
+ ;
+
+partitionsDef
+ : partitions+=partitionDef (COMMA partitions+=partitionDef)*
+ ;
+
+partitionDef
+ : lessThanPartitionDef
+ | fixedPartitionDef
+ | stepPartitionDef
+ | inPartitionDef
+ ;
+
+lessThanPartitionDef
+ : PARTITION partitionName=identifier VALUES LESS THAN (MAXVALUE | constantSeq)
Review Comment:
miss partition properties
##########
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4:
##########
@@ -40,6 +40,16 @@ statement
AS type=(RESTRICTIVE | PERMISSIVE)
TO (user=userIdentify | ROLE roleName=identifier)
USING LEFT_PAREN booleanExpression RIGHT_PAREN #createRowPolicy
+ | CREATE TABLE (IF NOT EXISTS)? name=multipartIdentifier
+ ((LEFT_PAREN columnDefs indexDefs? RIGHT_PAREN) | (ctasCols=identifierList)?)
+ (ENGINE EQ engine=identifier)?
+ ((AGGREGATE | UNIQUE | DUPLICATE) KEY keys=identifierList)?
+ (COMMENT constant)?
+ (PARTITION BY (RANGE | LIST) partitionKeys=identifierList LEFT_PAREN partitions=partitionsDef RIGHT_PAREN)?
+ DISTRIBUTED BY (HASH hashKeys=identifierList | RANDOM) BUCKETS (number | AUTO)?
Review Comment:
could not use `number`, should use `INTEGER_VALUE`
##########
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4:
##########
@@ -315,6 +325,72 @@ tableAlias
multipartIdentifier
: parts+=errorCapturingIdentifier (DOT parts+=errorCapturingIdentifier)*
;
+
+// ----------------Create Table Fields----------
+
+columnDefs
+ : cols+=columnDef (COMMA cols+=columnDef)*
+ ;
+
+columnDef
+ : colName=identifier type=dataType
+ KEY? (aggType=aggTypeDef)? ((NOT NULL) | NULL)?
+ (DEFAULT (defaultValue=constant | CURRENT_TIMESTAMP))?
+ (COMMENT comment=constant)?
Review Comment:
comment should not be `constant`
##########
fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4:
##########
@@ -315,6 +325,72 @@ tableAlias
multipartIdentifier
: parts+=errorCapturingIdentifier (DOT parts+=errorCapturingIdentifier)*
;
+
+// ----------------Create Table Fields----------
+
+columnDefs
+ : cols+=columnDef (COMMA cols+=columnDef)*
+ ;
+
+columnDef
+ : colName=identifier type=dataType
+ KEY? (aggType=aggTypeDef)? ((NOT NULL) | NULL)?
+ (DEFAULT (defaultValue=constant | CURRENT_TIMESTAMP))?
+ (COMMENT comment=constant)?
+ ;
+
+indexDefs
+ : indexes+=indexDef (COMMA indexes+=indexDef)*
+ ;
+
+indexDef
+ : INDEX indexName=identifier cols=identifierList (USING BITMAP)? (comment=constant)?
+ ;
+
+partitionsDef
+ : partitions+=partitionDef (COMMA partitions+=partitionDef)*
+ ;
+
+partitionDef
+ : lessThanPartitionDef
+ | fixedPartitionDef
+ | stepPartitionDef
+ | inPartitionDef
+ ;
+
+lessThanPartitionDef
+ : PARTITION partitionName=identifier VALUES LESS THAN (MAXVALUE | constantSeq)
+ ;
+
+fixedPartitionDef
+ : PARTITION partitionName=identifier VALUES LEFT_BRACKET lower=constantSeq COMMA upper=constantSeq RIGHT_PAREN
+ ;
+
+stepPartitionDef
+ : FROM from=constantSeq TO to=constantSeq
+ ((INTERVAL unitsAmount=valueExpression unit=datetimeUnit) | unitsAmount=valueExpression)
Review Comment:
should not use `valueExpression`, should use `INTEGER_LITERAL`
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -1718,6 +1745,194 @@ private LogicalPlan plan(ParserRuleContext tree) {
return (LogicalPlan) tree.accept(this);
}
+ /* ********************************************************************************************
+ * create table parsing
+ * ******************************************************************************************** */
+
+ @Override
+ public LogicalPlan visitCreateTable(CreateTableContext ctx) {
+ String dbName = null;
+ String tableName;
+ List<String> nameParts = visitMultipartIdentifier(ctx.name);
+ if (nameParts.size() == 1) {
+ tableName = nameParts.get(0);
+ } else if (nameParts.size() == 2) {
+ dbName = nameParts.get(0);
+ tableName = nameParts.get(1);
+ } else {
+ throw new AnalysisException("nameParts in create table should be 1 or 2");
+ }
+ KeysType keysType = null;
+ if (ctx.DUPLICATE() != null) {
+ keysType = KeysType.DUP_KEYS;
+ } else if (ctx.AGGREGATE() != null) {
+ keysType = KeysType.AGG_KEYS;
+ } else if (ctx.UNIQUE() != null) {
+ keysType = KeysType.UNIQUE_KEYS;
+ }
+ String engineName = ctx.engine != null ? ctx.engine.getText().toLowerCase() : "olap";
+ DistributionDescriptor desc = new DistributionDescriptor(ctx.HASH() != null, ctx.AUTO() != null,
+ Integer.parseInt(ctx.number().getText()),
+ ctx.HASH() != null ? visitIdentifierList(ctx.hashKeys) : null);
+ Map<String, String> properties = ctx.propertyClause() != null
+ ? visitPropertyClause(ctx.propertyClause()) : null;
Review Comment:
should not use null, use empty map
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -1718,6 +1745,194 @@ private LogicalPlan plan(ParserRuleContext tree) {
return (LogicalPlan) tree.accept(this);
}
+ /* ********************************************************************************************
+ * create table parsing
+ * ******************************************************************************************** */
+
+ @Override
+ public LogicalPlan visitCreateTable(CreateTableContext ctx) {
+ String dbName = null;
+ String tableName;
+ List<String> nameParts = visitMultipartIdentifier(ctx.name);
Review Comment:
should support catalog , `ctl.db.tbl`
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateTableCommand.java:
##########
@@ -0,0 +1,134 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.commands;
+
+import org.apache.doris.analysis.CreateTableStmt;
+import org.apache.doris.analysis.DropTableStmt;
+import org.apache.doris.analysis.TableName;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.nereids.NereidsPlanner;
+import org.apache.doris.nereids.analyzer.UnboundOlapTableSink;
+import org.apache.doris.nereids.annotation.Developing;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.properties.PhysicalProperties;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.PlanType;
+import org.apache.doris.nereids.trees.plans.commands.ExplainCommand.ExplainLevel;
+import org.apache.doris.nereids.trees.plans.commands.info.ColumnDefinition;
+import org.apache.doris.nereids.trees.plans.commands.info.CreateTableInfo;
+import org.apache.doris.nereids.trees.plans.logical.LogicalPlan;
+import org.apache.doris.nereids.trees.plans.visitor.PlanVisitor;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.QueryState.MysqlStateType;
+import org.apache.doris.qe.StmtExecutor;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+/**
+ * create table command
+ */
+@Developing
+public class CreateTableCommand extends Command implements ForwardWithSync {
+ public static final Logger LOG = LogManager.getLogger(CreateTableCommand.class);
+
+ private final Optional<LogicalPlan> ctasQuery;
+ private final CreateTableInfo createTableInfo;
+
+ public CreateTableCommand(Optional<LogicalPlan> ctasQuery, CreateTableInfo createTableInfo) {
+ super(PlanType.CREATE_TABLE_COMMAND);
+ this.ctasQuery = ctasQuery;
+ this.createTableInfo = Objects.requireNonNull(createTableInfo);
+ }
+
+ @Override
+ public void run(ConnectContext ctx, StmtExecutor executor) throws Exception {
+ if (!ctasQuery.isPresent()) {
+ createTableInfo.validate(ctx);
+ CreateTableStmt createTableStmt = createTableInfo.translateToCatalogStyle();
+ LOG.info("Nereids start to execute the create table command, query id: {}, tableName: {}",
Review Comment:
do not use info log, use debug
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateTableCommand.java:
##########
@@ -0,0 +1,134 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.commands;
+
+import org.apache.doris.analysis.CreateTableStmt;
+import org.apache.doris.analysis.DropTableStmt;
+import org.apache.doris.analysis.TableName;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.nereids.NereidsPlanner;
+import org.apache.doris.nereids.analyzer.UnboundOlapTableSink;
+import org.apache.doris.nereids.annotation.Developing;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.properties.PhysicalProperties;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.PlanType;
+import org.apache.doris.nereids.trees.plans.commands.ExplainCommand.ExplainLevel;
+import org.apache.doris.nereids.trees.plans.commands.info.ColumnDefinition;
+import org.apache.doris.nereids.trees.plans.commands.info.CreateTableInfo;
+import org.apache.doris.nereids.trees.plans.logical.LogicalPlan;
+import org.apache.doris.nereids.trees.plans.visitor.PlanVisitor;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.QueryState.MysqlStateType;
+import org.apache.doris.qe.StmtExecutor;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+/**
+ * create table command
+ */
+@Developing
+public class CreateTableCommand extends Command implements ForwardWithSync {
+ public static final Logger LOG = LogManager.getLogger(CreateTableCommand.class);
+
+ private final Optional<LogicalPlan> ctasQuery;
+ private final CreateTableInfo createTableInfo;
+
+ public CreateTableCommand(Optional<LogicalPlan> ctasQuery, CreateTableInfo createTableInfo) {
+ super(PlanType.CREATE_TABLE_COMMAND);
+ this.ctasQuery = ctasQuery;
+ this.createTableInfo = Objects.requireNonNull(createTableInfo);
Review Comment:
add check msg
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -1718,6 +1745,194 @@ private LogicalPlan plan(ParserRuleContext tree) {
return (LogicalPlan) tree.accept(this);
}
+ /* ********************************************************************************************
+ * create table parsing
+ * ******************************************************************************************** */
+
+ @Override
+ public LogicalPlan visitCreateTable(CreateTableContext ctx) {
+ String dbName = null;
+ String tableName;
+ List<String> nameParts = visitMultipartIdentifier(ctx.name);
+ if (nameParts.size() == 1) {
+ tableName = nameParts.get(0);
+ } else if (nameParts.size() == 2) {
+ dbName = nameParts.get(0);
+ tableName = nameParts.get(1);
+ } else {
+ throw new AnalysisException("nameParts in create table should be 1 or 2");
+ }
+ KeysType keysType = null;
+ if (ctx.DUPLICATE() != null) {
+ keysType = KeysType.DUP_KEYS;
+ } else if (ctx.AGGREGATE() != null) {
+ keysType = KeysType.AGG_KEYS;
+ } else if (ctx.UNIQUE() != null) {
+ keysType = KeysType.UNIQUE_KEYS;
+ }
+ String engineName = ctx.engine != null ? ctx.engine.getText().toLowerCase() : "olap";
+ DistributionDescriptor desc = new DistributionDescriptor(ctx.HASH() != null, ctx.AUTO() != null,
+ Integer.parseInt(ctx.number().getText()),
+ ctx.HASH() != null ? visitIdentifierList(ctx.hashKeys) : null);
+ Map<String, String> properties = ctx.propertyClause() != null
+ ? visitPropertyClause(ctx.propertyClause()) : null;
+ String partitionType = null;
+ if (ctx.PARTITION() != null) {
+ partitionType = ctx.RANGE() != null ? "RANGE" : "LIST";
+ }
+
+ if (ctx.columnDefs() != null) {
+ if (ctx.AS() != null) {
+ throw new AnalysisException("Should not define the entire column in CTAS");
+ }
+ return new CreateTableCommand(Optional.empty(), new CreateTableInfo(
+ ctx.EXISTS() != null,
+ dbName,
+ tableName,
+ visitColumnDefs(ctx.columnDefs()),
+ ImmutableList.of(),
+ engineName,
+ keysType,
+ ctx.keys != null ? visitIdentifierList(ctx.keys) : ImmutableList.of(),
+ "",
+ partitionType,
+ ctx.partitionKeys != null ? visitIdentifierList(ctx.partitionKeys) : null,
+ ctx.partitions != null ? visitPartitionsDef(ctx.partitions) : null,
+ desc,
+ ImmutableList.of(),
+ properties));
+ } else if (ctx.AS() != null) {
+ return new CreateTableCommand(Optional.of(visitQuery(ctx.query())), new CreateTableInfo(
+ ctx.EXISTS() != null,
+ dbName,
+ tableName,
+ ctx.ctasCols != null ? visitIdentifierList(ctx.ctasCols) : null,
+ engineName,
+ keysType,
+ ctx.keys != null ? visitIdentifierList(ctx.keys) : ImmutableList.of(),
+ "",
+ partitionType,
+ ctx.partitionKeys != null ? visitIdentifierList(ctx.partitionKeys) : null,
+ ctx.partitions != null ? visitPartitionsDef(ctx.partitions) : null,
+ desc,
+ ImmutableList.of(),
+ properties));
+ } else {
+ throw new AnalysisException("Should contain at least one column in a table");
+ }
+ }
+
+ @Override
+ public List<ColumnDefinition> visitColumnDefs(ColumnDefsContext ctx) {
+ return ctx.cols.stream().map(this::visitColumnDef).collect(Collectors.toList());
+ }
+
+ @Override
+ public ColumnDefinition visitColumnDef(ColumnDefContext ctx) {
+ String colName = ctx.colName.getText();
+ DataType colType = ctx.type instanceof PrimitiveDataTypeContext
+ ? visitPrimitiveDataType(((PrimitiveDataTypeContext) ctx.type))
+ : visitComplexDataType(((ComplexDataTypeContext) ctx.type));
+ boolean isKey = ctx.KEY() != null;
+ boolean isNotNull = ctx.NOT() != null;
+ String aggTypeString = ctx.aggType != null ? ctx.aggType.getText() : null;
+ Optional<Expression> defaultValue = Optional.empty();
+ if (ctx.DEFAULT() != null) {
+ if (ctx.defaultValue != null) {
+ defaultValue = Optional.of(((Literal) visit(ctx.defaultValue)));
+ } else if (ctx.CURRENT_TIMESTAMP() != null) {
+ defaultValue = Optional.of(new Now());
+ }
+ }
+ AggregateType aggType = null;
+ if (aggTypeString != null) {
+ try {
+ aggType = AggregateType.valueOf(aggTypeString.toUpperCase());
+ } catch (Exception e) {
+ throw new AnalysisException(String.format("Aggregate type %s is unsupported", aggTypeString),
+ e.getCause());
+ }
+ }
+ String comment = ctx.comment != null ? ((Literal) visit(ctx.comment)).getStringValue() : "";
+ return new ColumnDefinition(colName, colType, isKey, aggType, !isNotNull, defaultValue, comment);
+ }
+
+ @Override
+ public List<IndexDefinition> visitIndexDefs(IndexDefsContext ctx) {
+ return ctx.indexes.stream().map(this::visitIndexDef).collect(Collectors.toList());
+ }
+
+ @Override
+ public IndexDefinition visitIndexDef(IndexDefContext ctx) {
+ String indexName = ctx.indexName.getText();
+ List<String> indexCols = visitIdentifierList(ctx.cols);
+ boolean isUseBitmap = ctx.USING() != null;
+ String comment = ((Literal) visit(ctx.comment)).getStringValue();
+ return new IndexDefinition(indexName, indexCols, isUseBitmap, comment);
+ }
+
+ @Override
+ public List<PartitionDefinition> visitPartitionsDef(PartitionsDefContext ctx) {
+ return ctx.partitions.stream().map(p -> ((PartitionDefinition) visit(p))).collect(Collectors.toList());
Review Comment:
ImmutableList
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -1718,6 +1745,194 @@ private LogicalPlan plan(ParserRuleContext tree) {
return (LogicalPlan) tree.accept(this);
}
+ /* ********************************************************************************************
+ * create table parsing
+ * ******************************************************************************************** */
+
+ @Override
+ public LogicalPlan visitCreateTable(CreateTableContext ctx) {
+ String dbName = null;
+ String tableName;
+ List<String> nameParts = visitMultipartIdentifier(ctx.name);
+ if (nameParts.size() == 1) {
+ tableName = nameParts.get(0);
+ } else if (nameParts.size() == 2) {
+ dbName = nameParts.get(0);
+ tableName = nameParts.get(1);
+ } else {
+ throw new AnalysisException("nameParts in create table should be 1 or 2");
+ }
+ KeysType keysType = null;
+ if (ctx.DUPLICATE() != null) {
+ keysType = KeysType.DUP_KEYS;
+ } else if (ctx.AGGREGATE() != null) {
+ keysType = KeysType.AGG_KEYS;
+ } else if (ctx.UNIQUE() != null) {
+ keysType = KeysType.UNIQUE_KEYS;
+ }
+ String engineName = ctx.engine != null ? ctx.engine.getText().toLowerCase() : "olap";
+ DistributionDescriptor desc = new DistributionDescriptor(ctx.HASH() != null, ctx.AUTO() != null,
+ Integer.parseInt(ctx.number().getText()),
+ ctx.HASH() != null ? visitIdentifierList(ctx.hashKeys) : null);
+ Map<String, String> properties = ctx.propertyClause() != null
+ ? visitPropertyClause(ctx.propertyClause()) : null;
+ String partitionType = null;
+ if (ctx.PARTITION() != null) {
+ partitionType = ctx.RANGE() != null ? "RANGE" : "LIST";
+ }
+
+ if (ctx.columnDefs() != null) {
+ if (ctx.AS() != null) {
+ throw new AnalysisException("Should not define the entire column in CTAS");
+ }
+ return new CreateTableCommand(Optional.empty(), new CreateTableInfo(
+ ctx.EXISTS() != null,
+ dbName,
+ tableName,
+ visitColumnDefs(ctx.columnDefs()),
+ ImmutableList.of(),
+ engineName,
+ keysType,
+ ctx.keys != null ? visitIdentifierList(ctx.keys) : ImmutableList.of(),
+ "",
+ partitionType,
+ ctx.partitionKeys != null ? visitIdentifierList(ctx.partitionKeys) : null,
+ ctx.partitions != null ? visitPartitionsDef(ctx.partitions) : null,
+ desc,
+ ImmutableList.of(),
+ properties));
+ } else if (ctx.AS() != null) {
+ return new CreateTableCommand(Optional.of(visitQuery(ctx.query())), new CreateTableInfo(
+ ctx.EXISTS() != null,
+ dbName,
+ tableName,
+ ctx.ctasCols != null ? visitIdentifierList(ctx.ctasCols) : null,
+ engineName,
+ keysType,
+ ctx.keys != null ? visitIdentifierList(ctx.keys) : ImmutableList.of(),
+ "",
+ partitionType,
+ ctx.partitionKeys != null ? visitIdentifierList(ctx.partitionKeys) : null,
+ ctx.partitions != null ? visitPartitionsDef(ctx.partitions) : null,
+ desc,
+ ImmutableList.of(),
+ properties));
+ } else {
+ throw new AnalysisException("Should contain at least one column in a table");
+ }
+ }
+
+ @Override
+ public List<ColumnDefinition> visitColumnDefs(ColumnDefsContext ctx) {
+ return ctx.cols.stream().map(this::visitColumnDef).collect(Collectors.toList());
+ }
+
+ @Override
+ public ColumnDefinition visitColumnDef(ColumnDefContext ctx) {
+ String colName = ctx.colName.getText();
+ DataType colType = ctx.type instanceof PrimitiveDataTypeContext
+ ? visitPrimitiveDataType(((PrimitiveDataTypeContext) ctx.type))
+ : visitComplexDataType(((ComplexDataTypeContext) ctx.type));
+ boolean isKey = ctx.KEY() != null;
+ boolean isNotNull = ctx.NOT() != null;
+ String aggTypeString = ctx.aggType != null ? ctx.aggType.getText() : null;
+ Optional<Expression> defaultValue = Optional.empty();
+ if (ctx.DEFAULT() != null) {
+ if (ctx.defaultValue != null) {
+ defaultValue = Optional.of(((Literal) visit(ctx.defaultValue)));
+ } else if (ctx.CURRENT_TIMESTAMP() != null) {
+ defaultValue = Optional.of(new Now());
+ }
+ }
+ AggregateType aggType = null;
+ if (aggTypeString != null) {
+ try {
+ aggType = AggregateType.valueOf(aggTypeString.toUpperCase());
+ } catch (Exception e) {
+ throw new AnalysisException(String.format("Aggregate type %s is unsupported", aggTypeString),
+ e.getCause());
+ }
+ }
+ String comment = ctx.comment != null ? ((Literal) visit(ctx.comment)).getStringValue() : "";
+ return new ColumnDefinition(colName, colType, isKey, aggType, !isNotNull, defaultValue, comment);
+ }
+
+ @Override
+ public List<IndexDefinition> visitIndexDefs(IndexDefsContext ctx) {
+ return ctx.indexes.stream().map(this::visitIndexDef).collect(Collectors.toList());
Review Comment:
ImmutableList
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateTableCommand.java:
##########
@@ -0,0 +1,134 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.commands;
+
+import org.apache.doris.analysis.CreateTableStmt;
+import org.apache.doris.analysis.DropTableStmt;
+import org.apache.doris.analysis.TableName;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.nereids.NereidsPlanner;
+import org.apache.doris.nereids.analyzer.UnboundOlapTableSink;
+import org.apache.doris.nereids.annotation.Developing;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.properties.PhysicalProperties;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.PlanType;
+import org.apache.doris.nereids.trees.plans.commands.ExplainCommand.ExplainLevel;
+import org.apache.doris.nereids.trees.plans.commands.info.ColumnDefinition;
+import org.apache.doris.nereids.trees.plans.commands.info.CreateTableInfo;
+import org.apache.doris.nereids.trees.plans.logical.LogicalPlan;
+import org.apache.doris.nereids.trees.plans.visitor.PlanVisitor;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.QueryState.MysqlStateType;
+import org.apache.doris.qe.StmtExecutor;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+/**
+ * create table command
+ */
+@Developing
+public class CreateTableCommand extends Command implements ForwardWithSync {
+ public static final Logger LOG = LogManager.getLogger(CreateTableCommand.class);
+
+ private final Optional<LogicalPlan> ctasQuery;
+ private final CreateTableInfo createTableInfo;
+
+ public CreateTableCommand(Optional<LogicalPlan> ctasQuery, CreateTableInfo createTableInfo) {
+ super(PlanType.CREATE_TABLE_COMMAND);
+ this.ctasQuery = ctasQuery;
+ this.createTableInfo = Objects.requireNonNull(createTableInfo);
+ }
+
+ @Override
+ public void run(ConnectContext ctx, StmtExecutor executor) throws Exception {
+ if (!ctasQuery.isPresent()) {
+ createTableInfo.validate(ctx);
+ CreateTableStmt createTableStmt = createTableInfo.translateToCatalogStyle();
+ LOG.info("Nereids start to execute the create table command, query id: {}, tableName: {}",
+ ctx.queryId(), createTableInfo.getTableName());
+ try {
+ Env.getCurrentEnv().createTable(createTableStmt);
+ } catch (Exception e) {
+ throw new AnalysisException(e.getMessage(), e.getCause());
+ }
+ return;
+ }
+ LogicalPlan query = ctasQuery.get();
+ List<String> ctasCols = createTableInfo.getCtasColumns();
+ NereidsPlanner planner = new NereidsPlanner(ctx.getStatementContext());
+ Plan plan = planner.plan(query, PhysicalProperties.ANY, ExplainLevel.NONE);
+ if (ctasCols == null) {
+ // we should analyze the plan firstly to get the columns' name.
+ ctasCols = plan.getOutput().stream().map(NamedExpression::getName).collect(Collectors.toList());
+ }
+ List<Slot> slots = plan.getOutput();
+ if (slots.size() != ctasCols.size()) {
+ throw new AnalysisException("ctas column size is not equal to the query's");
+ }
+ List<ColumnDefinition> columnsOfQuery = slots.stream()
+ .map(s -> new ColumnDefinition(s.getName(), s.getDataType(), s.nullable()))
+ .collect(Collectors.toList());
+ createTableInfo.validateCreateTableAsSelect(columnsOfQuery, ctx);
+
+ CreateTableStmt createTableStmt = createTableInfo.translateToCatalogStyle();
Review Comment:
```suggestion
CreateTableStmt createTableStmt = createTableInfo.translateToLegacyStmt();
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java:
##########
@@ -0,0 +1,388 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.commands.info;
+
+import org.apache.doris.analysis.AllPartitionDesc;
+import org.apache.doris.analysis.CreateTableStmt;
+import org.apache.doris.analysis.KeysDesc;
+import org.apache.doris.analysis.ListPartitionDesc;
+import org.apache.doris.analysis.PartitionDesc;
+import org.apache.doris.analysis.RangePartitionDesc;
+import org.apache.doris.analysis.TableName;
+import org.apache.doris.catalog.AggregateType;
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.Index;
+import org.apache.doris.catalog.KeysType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.cluster.ClusterNamespace;
+import org.apache.doris.common.Config;
+import org.apache.doris.common.FeConstants;
+import org.apache.doris.common.util.PropertyAnalyzer;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.nereids.util.Utils;
+import org.apache.doris.qe.ConnectContext;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * table info in creating table.
+ */
+public class CreateTableInfo {
+ private final boolean ifNotExists;
+ private String dbName;
+ private final String tableName;
+ private List<ColumnDefinition> columns;
+ private final List<IndexDefinition> indexes;
+ private final List<String> ctasColumns;
+ private final String engineName;
+ private KeysType keysType;
+ private List<String> keys;
+ private final String comment;
+ private final String partitionType;
+ private final List<String> partitionColumns;
+ private final List<PartitionDefinition> partitions;
+ private final DistributionDescriptor distribution;
+ private final List<RollupDefinition> rollups;
+ private Map<String, String> properties;
+
+ /**
+ * constructor for create table
+ */
+ public CreateTableInfo(boolean ifNotExists, String dbName, String tableName, List<ColumnDefinition> columns,
+ List<IndexDefinition> indexes, String engineName, KeysType keysType, List<String> keys, String comment,
+ String partitionType, List<String> partitionColumns, List<PartitionDefinition> partitions,
+ DistributionDescriptor distribution, List<RollupDefinition> rollups, Map<String, String> properties) {
+ this.ifNotExists = ifNotExists;
+ this.dbName = dbName;
+ this.tableName = tableName;
+ this.ctasColumns = null;
+ this.columns = Utils.copyRequiredMutableList(columns);
+ this.indexes = Utils.copyRequiredList(indexes);
+ this.engineName = engineName;
+ this.keysType = keysType;
+ this.keys = Utils.copyRequiredList(keys);
+ this.comment = comment;
+ this.partitionType = partitionType;
+ this.partitionColumns = partitionColumns;
+ this.partitions = partitions;
+ this.distribution = distribution;
+ this.rollups = Utils.copyRequiredList(rollups);
+ this.properties = properties;
+ }
+
+ /**
+ * constructor for create table as select
+ */
+ public CreateTableInfo(boolean ifNotExists, String dbName, String tableName, List<String> cols,
+ String engineName, KeysType keysType, List<String> keys, String comment,
+ String partitionType, List<String> partitionColumns, List<PartitionDefinition> partitions,
+ DistributionDescriptor distribution, List<RollupDefinition> rollups, Map<String, String> properties) {
+ this.ifNotExists = ifNotExists;
+ this.dbName = dbName;
+ this.tableName = tableName;
+ this.ctasColumns = cols;
+ this.columns = ImmutableList.of();
+ this.indexes = ImmutableList.of();
+ this.engineName = engineName;
+ this.keysType = keysType;
+ this.keys = Utils.copyRequiredList(keys);
+ this.comment = comment;
+ this.partitionType = partitionType;
+ this.partitionColumns = partitionColumns;
+ this.partitions = partitions;
+ this.distribution = distribution;
+ this.rollups = Utils.copyRequiredList(rollups);
+ this.properties = properties;
+ }
+
+ public List<String> getCtasColumns() {
+ return ctasColumns;
+ }
+
+ public String getDbName() {
+ return dbName;
+ }
+
+ public String getTableName() {
+ return tableName;
+ }
+
+ public List<String> getTableNameParts() {
+ if (dbName != null) {
+ return ImmutableList.of(dbName, tableName);
+ }
+ return ImmutableList.of(tableName);
+ }
+
+ /**
+ * analyze create table info
+ */
+ public void validate(ConnectContext ctx) {
+ // pre-block in some cases.
+ if (columns.isEmpty()) {
+ throw new AnalysisException("table should contain at least one column");
+ }
+ if (distribution == null) {
+ throw new AnalysisException("Create olap table should contain distribution desc");
+ }
+ if (!engineName.equals("olap")) {
Review Comment:
ingoreCaseEquals
##########
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/CreateTableCommandTest.java:
##########
Review Comment:
i think u should test all CreateTableInfo class interface directly in UT. such as call xxx.validate directly, call translateToLegacyStmt directly. sql test should put into regression test
##########
regression-test/suites/nereids_p0/create_table/test_create_table.groovy:
##########
@@ -0,0 +1,56 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+// this suite is for creating table with timestamp datatype in defferent
+// case. For example: 'year' and 'Year' datatype should also be valid in definition
+
+suite("nereids_create_time_range_table") {
+ sql 'set enable_nereids_planner=true'
+ sql 'set enable_fallback_to_original_planner=false'
+
+ def testTable = "test_time_range_table"
+
+ sql "DROP TABLE IF EXISTS ${testTable}"
+
+ // multi-line sql
+ def result1 = sql """
+ create table ${testTable} (
+ `actorid` varchar(128),
+ `gameid` varchar(128),
+ `eventtime` datetimev2(3)
+ )
+ engine=olap
Review Comment:
should cover all engine type, key type, index type, rollup type, partition def, column type, agg key type, etc.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java:
##########
@@ -0,0 +1,388 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.commands.info;
+
+import org.apache.doris.analysis.AllPartitionDesc;
+import org.apache.doris.analysis.CreateTableStmt;
+import org.apache.doris.analysis.KeysDesc;
+import org.apache.doris.analysis.ListPartitionDesc;
+import org.apache.doris.analysis.PartitionDesc;
+import org.apache.doris.analysis.RangePartitionDesc;
+import org.apache.doris.analysis.TableName;
+import org.apache.doris.catalog.AggregateType;
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.Index;
+import org.apache.doris.catalog.KeysType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.cluster.ClusterNamespace;
+import org.apache.doris.common.Config;
+import org.apache.doris.common.FeConstants;
+import org.apache.doris.common.util.PropertyAnalyzer;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.nereids.util.Utils;
+import org.apache.doris.qe.ConnectContext;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * table info in creating table.
+ */
+public class CreateTableInfo {
+ private final boolean ifNotExists;
+ private String dbName;
+ private final String tableName;
+ private List<ColumnDefinition> columns;
+ private final List<IndexDefinition> indexes;
+ private final List<String> ctasColumns;
+ private final String engineName;
+ private KeysType keysType;
+ private List<String> keys;
+ private final String comment;
+ private final String partitionType;
+ private final List<String> partitionColumns;
+ private final List<PartitionDefinition> partitions;
+ private final DistributionDescriptor distribution;
+ private final List<RollupDefinition> rollups;
+ private Map<String, String> properties;
+
+ /**
+ * constructor for create table
+ */
+ public CreateTableInfo(boolean ifNotExists, String dbName, String tableName, List<ColumnDefinition> columns,
+ List<IndexDefinition> indexes, String engineName, KeysType keysType, List<String> keys, String comment,
+ String partitionType, List<String> partitionColumns, List<PartitionDefinition> partitions,
+ DistributionDescriptor distribution, List<RollupDefinition> rollups, Map<String, String> properties) {
+ this.ifNotExists = ifNotExists;
+ this.dbName = dbName;
+ this.tableName = tableName;
+ this.ctasColumns = null;
+ this.columns = Utils.copyRequiredMutableList(columns);
+ this.indexes = Utils.copyRequiredList(indexes);
+ this.engineName = engineName;
+ this.keysType = keysType;
+ this.keys = Utils.copyRequiredList(keys);
+ this.comment = comment;
+ this.partitionType = partitionType;
+ this.partitionColumns = partitionColumns;
+ this.partitions = partitions;
+ this.distribution = distribution;
+ this.rollups = Utils.copyRequiredList(rollups);
+ this.properties = properties;
+ }
+
+ /**
+ * constructor for create table as select
+ */
+ public CreateTableInfo(boolean ifNotExists, String dbName, String tableName, List<String> cols,
+ String engineName, KeysType keysType, List<String> keys, String comment,
+ String partitionType, List<String> partitionColumns, List<PartitionDefinition> partitions,
+ DistributionDescriptor distribution, List<RollupDefinition> rollups, Map<String, String> properties) {
+ this.ifNotExists = ifNotExists;
+ this.dbName = dbName;
+ this.tableName = tableName;
+ this.ctasColumns = cols;
+ this.columns = ImmutableList.of();
+ this.indexes = ImmutableList.of();
+ this.engineName = engineName;
+ this.keysType = keysType;
+ this.keys = Utils.copyRequiredList(keys);
+ this.comment = comment;
+ this.partitionType = partitionType;
+ this.partitionColumns = partitionColumns;
+ this.partitions = partitions;
+ this.distribution = distribution;
+ this.rollups = Utils.copyRequiredList(rollups);
+ this.properties = properties;
+ }
+
+ public List<String> getCtasColumns() {
+ return ctasColumns;
+ }
+
+ public String getDbName() {
+ return dbName;
+ }
+
+ public String getTableName() {
+ return tableName;
+ }
+
+ public List<String> getTableNameParts() {
+ if (dbName != null) {
+ return ImmutableList.of(dbName, tableName);
+ }
+ return ImmutableList.of(tableName);
+ }
+
+ /**
+ * analyze create table info
+ */
+ public void validate(ConnectContext ctx) {
+ // pre-block in some cases.
+ if (columns.isEmpty()) {
+ throw new AnalysisException("table should contain at least one column");
+ }
+ if (distribution == null) {
+ throw new AnalysisException("Create olap table should contain distribution desc");
+ }
+ if (!engineName.equals("olap")) {
+ throw new AnalysisException("currently Nereids support olap engine only");
+ }
+ if (properties == null) {
+ properties = Maps.newHashMap();
+ }
+
+ // analyze table name
+ if (dbName == null) {
+ dbName = ClusterNamespace.getFullName(ctx.getClusterName(), ctx.getDatabase());
+ } else {
+ dbName = ClusterNamespace.getFullName(ctx.getClusterName(), dbName);
+ }
+
+ boolean isEnableMergeOnWrite = false;
+ boolean enableDuplicateWithoutKeysByDefault = false;
+ if (properties != null) {
+ try {
+ enableDuplicateWithoutKeysByDefault =
+ PropertyAnalyzer.analyzeEnableDuplicateWithoutKeysByDefault(Maps.newHashMap(properties));
+ } catch (Exception e) {
+ throw new AnalysisException(e.getMessage(), e.getCause());
+ }
+ }
+
+ if (keys.isEmpty()) {
+ boolean hasAggColumn = false;
+ for (ColumnDefinition column : columns) {
+ if (column.getAggType() != null) {
+ hasAggColumn = true;
+ break;
+ }
+ }
+ keys = Lists.newArrayList();
+ if (hasAggColumn) {
+ for (ColumnDefinition column : columns) {
+ if (column.getAggType() != null) {
+ break;
+ }
+ keys.add(column.getName());
+ }
+ keysType = KeysType.AGG_KEYS;
+ } else {
+ int keyLength = 0;
+ for (ColumnDefinition column : columns) {
+ DataType type = column.getType();
+ Type catalogType = column.getType().toCatalogDataType();
+ keyLength += catalogType.getIndexSize();
+ if (keys.size() >= FeConstants.shortkey_max_column_count
+ || keyLength > FeConstants.shortkey_maxsize_bytes) {
+ if (keys.size() == 0 && type.isStringLikeType()) {
+ keys.add(column.getName());
+ }
+ break;
+ }
+ if (type.isFloatLikeType() || type.isStringType() || type.isJsonType()
+ || catalogType.isComplexType()) {
+ break;
+ }
+ keys.add(column.getName());
+ if (type.isVarcharType()) {
+ break;
+ }
+ }
+ keysType = KeysType.DUP_KEYS;
+ }
+ // The OLAP table must have at least one short key,
+ // and the float and double should not be short key,
+ // so the float and double could not be the first column in OLAP table.
+ if (keys.isEmpty()) {
+ throw new AnalysisException("The olap table first column could not be float, double, string"
+ + " or array, struct, map, please use decimal or varchar instead.");
+ }
+ } else if (enableDuplicateWithoutKeysByDefault) {
+ throw new AnalysisException("table property 'enable_duplicate_without_keys_by_default' only can"
+ + " set 'true' when create olap table by default.");
+ }
+
+ if (keysType.equals(KeysType.UNIQUE_KEYS) && properties != null) {
+ try {
+ isEnableMergeOnWrite = PropertyAnalyzer.analyzeUniqueKeyMergeOnWrite(Maps.newHashMap(properties));
+ } catch (Exception e) {
+ throw new AnalysisException(e.getMessage(), e.getCause());
+ }
+ }
+
+ // add hidden column
+ if (Config.enable_batch_delete_by_default && keysType.equals(KeysType.UNIQUE_KEYS)) {
+ if (isEnableMergeOnWrite) {
+ columns.add(ColumnDefinition.newDeleteSignColumnDefinition(AggregateType.NONE));
+ } else {
+ columns.add(ColumnDefinition.newDeleteSignColumnDefinition(AggregateType.REPLACE));
+ }
+ }
+
+ // add a hidden column as row store
+ boolean storeRowColumn = false;
+ if (properties != null) {
+ try {
+ storeRowColumn = PropertyAnalyzer.analyzeStoreRowColumn(Maps.newHashMap(properties));
+ } catch (Exception e) {
+ throw new AnalysisException(e.getMessage(), e.getCause());
+ }
+ }
+ if (storeRowColumn) {
+ if (keysType.equals(KeysType.AGG_KEYS)) {
+ throw new AnalysisException("Aggregate table can't support row column now");
+ }
+ if (keysType.equals(KeysType.UNIQUE_KEYS)) {
+ if (isEnableMergeOnWrite) {
+ columns.add(ColumnDefinition.newRowStoreColumnDefinition(AggregateType.NONE));
+ } else {
+ columns.add(ColumnDefinition.newRowStoreColumnDefinition(AggregateType.REPLACE));
+ }
+ } else {
+ columns.add(ColumnDefinition.newRowStoreColumnDefinition(null));
+ }
+ }
+ if (Config.enable_hidden_version_column_by_default && keysType.equals(KeysType.UNIQUE_KEYS)) {
+ if (isEnableMergeOnWrite) {
+ columns.add(ColumnDefinition.newVersionColumnDefinition(AggregateType.NONE));
+ } else {
+ columns.add(ColumnDefinition.newVersionColumnDefinition(AggregateType.REPLACE));
+ }
+ }
+
+ // analyze partitions
+ Map<String, ColumnDefinition> columnMap = columns.stream()
+ .collect(Collectors.toMap(ColumnDefinition::getName, c -> c));
+
+ if (partitions != null) {
+ if (!checkPartitionsTypes()) {
+ throw new AnalysisException("partitions types is invalid, expected FIXED or LESS in range partitions"
+ + " and IN in list partitions");
+ }
+ Set<String> partitionColumnSets = Sets.newHashSet(partitionColumns);
+ if (partitionColumnSets.size() != partitionColumns.size()) {
+ throw new AnalysisException("Duplicate partition keys is not allowed");
+ }
+ partitionColumns.forEach(c -> {
+ if (!columnMap.containsKey(c)) {
+ throw new AnalysisException(String.format("partition key %s is not found", c));
+ }
+ ColumnDefinition column = columnMap.get(c);
+ if (column.getType().isFloatLikeType()) {
+ throw new AnalysisException("Floating point type column can not be partition column");
+ }
+ });
+ partitions.forEach(p -> {
+ p.setPartitionTypes(partitionColumns.stream().map(s -> columnMap.get(s).getType())
+ .collect(Collectors.toList()));
+ p.validate(Maps.newHashMap(properties));
+ });
+ }
+
+ // analyze distribution descriptor
+ distribution.validate(columnMap, keysType);
+
+ final boolean finalEnableMergeOnWrite = isEnableMergeOnWrite;
+ Set<String> keysSet = Sets.newHashSet(keys);
+ columns.forEach(c -> c.validate(keysSet, finalEnableMergeOnWrite, keysType));
+
+ // analyze key set.
+ if (!distribution.isHash()) {
+ if (keysType.equals(KeysType.UNIQUE_KEYS)) {
+ throw new AnalysisException("Should not be distributed by random when keys type is unique");
+ } else if (keysType.equals(KeysType.AGG_KEYS)) {
+ for (ColumnDefinition c : columns) {
+ if (AggregateType.REPLACE.equals(c.getAggType())
+ || AggregateType.REPLACE_IF_NOT_NULL.equals(c.getAggType())) {
+ throw new AnalysisException("Should not be distributed by random when keys type is agg"
+ + "and column is in replace, [" + c.getName() + "] is invalid");
+ }
+ }
+ }
+ }
+ }
+
+ public void validateCreateTableAsSelect(List<ColumnDefinition> columns, ConnectContext ctx) {
+ this.columns = columns;
+ validate(ctx);
+ }
+
+ /**
+ * check partitions types.
+ */
+ public boolean checkPartitionsTypes() {
+ if (partitionType.equals("RANGE")) {
Review Comment:
all string equals should use ignoreCaseEquals
##########
regression-test/suites/nereids_p0/create_table/test_create_table.groovy:
##########
@@ -0,0 +1,56 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+// this suite is for creating table with timestamp datatype in defferent
+// case. For example: 'year' and 'Year' datatype should also be valid in definition
+
+suite("nereids_create_time_range_table") {
Review Comment:
suite name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22195: [Feature](Nereids)framework of create table and CTAS for Nereids.
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1705114460
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.62 seconds
stream load tsv: 534 seconds loaded 74807831229 Bytes, about 133 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162390797 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1661903404
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1664214084
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.38 seconds
stream load tsv: 499 seconds loaded 74807831229 Bytes, about 142 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 61 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 25.0 seconds inserted 10000000 Rows, about 400K ops/s
storage size: 17072071216 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1698685264
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1655722308
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)framework of create table and CTAS for Nereids.
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1705408053
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on a diff in pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on code in PR #22195:
URL: https://github.com/apache/doris/pull/22195#discussion_r1315509923
##########
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/CreateTableCommandTest.java:
##########
Review Comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1710245345
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.34 seconds
stream load tsv: 528 seconds loaded 74807831229 Bytes, about 135 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162058982 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712770425
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 50.52 seconds
stream load tsv: 583 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162426632 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)framework of create table and CTAS for Nereids.
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1705173929
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1689664447
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712530326
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712709289
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.5 seconds
stream load tsv: 584 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162359033 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1710192025
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1712817114
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1713037571
instead by this PR #24150
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1695422557
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1658499065
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.68 seconds
stream load tsv: 498 seconds loaded 74807831229 Bytes, about 143 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 25.3 seconds inserted 10000000 Rows, about 395K ops/s
storage size: 17072070649 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1697577935
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1698745223
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.55 seconds
stream load tsv: 529 seconds loaded 74807831229 Bytes, about 134 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 24.5 seconds inserted 10000000 Rows, about 408K ops/s
storage size: 17071999364 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids)support create table and CTAS for Nereids
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1695576878
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1709797505
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 50.22 seconds
stream load tsv: 540 seconds loaded 74807831229 Bytes, about 132 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162084794 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] selectdb-robot commented on pull request #22195: [Feature](Nereids)framework of create table and CTAS for Nereids.
Posted by "selectdb-robot (via GitHub)" <gi...@apache.org>.
selectdb-robot commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1705229503
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.49 seconds
stream load tsv: 530 seconds loaded 74807831229 Bytes, about 134 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17161916880 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1711519277
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on PR #22195:
URL: https://github.com/apache/doris/pull/22195#issuecomment-1708686862
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] sohardforaname commented on a diff in pull request #22195: [Feature](Nereids) support create table and CTAS
Posted by "sohardforaname (via GitHub)" <gi...@apache.org>.
sohardforaname commented on code in PR #22195:
URL: https://github.com/apache/doris/pull/22195#discussion_r1315537084
##########
regression-test/suites/nereids_p0/create_table/test_fnf.groovy:
##########
@@ -0,0 +1,86 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("nereids_test_fnf") {
Review Comment:
I don't know, it's the original suite name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org