You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "dutyu (via GitHub)" <gi...@apache.org> on 2023/12/22 11:10:50 UTC
[PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
dutyu opened a new pull request, #28890:
URL: https://github.com/apache/doris/pull/28890
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1881119901
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1874148140
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit d5541e4211d80ff8a7765a2d8fac08a14fdbc544, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5464 5120 5243 5120
q2 401 171 166 166
q3 1463 1235 1118 1118
q4 1111 831 787 787
q5 3021 3114 3163 3114
q6 228 143 132 132
q7 984 557 517 517
q8 2153 2266 2161 2161
q9 6708 6654 6661 6654
q10 3151 3111 3121 3111
q11 349 226 223 223
q12 375 240 237 237
q13 4459 3609 3618 3609
q14 253 212 211 211
q15 598 544 543 543
q16 450 405 417 405
q17 1040 555 536 536
q18 7057 6770 6784 6770
q19 1635 1520 1512 1512
q20 633 353 337 337
q21 2893 2460 2509 2460
q22 396 325 343 325
Total cold run time: 44822 ms
Total hot run time: 40048 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5219 5164 5176 5164
q2 340 243 236 236
q3 3393 3326 3309 3309
q4 2149 1999 1974 1974
q5 5916 5917 5911 5911
q6 229 128 125 125
q7 2386 1965 1889 1889
q8 3553 3643 3687 3643
q9 9078 9011 8978 8978
q10 3871 3911 3887 3887
q11 581 484 465 465
q12 808 638 631 631
q13 3893 3197 3173 3173
q14 299 282 263 263
q15 617 541 545 541
q16 563 512 491 491
q17 2039 1827 1801 1801
q18 8798 8324 8309 8309
q19 1743 1660 1669 1660
q20 2293 1981 1996 1981
q21 5703 5278 5324 5278
q22 567 510 498 498
Total cold run time: 64038 ms
Total hot run time: 60207 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1874107806
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1874881676
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools
```
TPC-DS sf100 test result on commit 5d8e5ed471bb0b5ca7f39bf1ef4dec1a2bc0d0f5, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 928 348 338 338
query2 5226 2056 2163 2056
query3 6584 205 201 201
query4 27204 22290 22394 22290
query5 4960 570 523 523
query6 272 182 180 180
query7 4544 276 278 276
query8 233 215 195 195
query9 8162 2590 2668 2590
query10 435 254 237 237
query11 16324 15493 15464 15464
query12 130 80 74 74
query13 1604 332 337 332
query14 11303 7267 7100 7100
query15 254 205 201 201
query16 6265 272 273 272
query17 1693 507 495 495
query18 1862 268 265 265
query19 273 141 135 135
query20 84 78 77 77
query21 170 97 92 92
query22 4975 4878 4481 4481
query23 32011 31151 31147 31147
query24 11990 2865 2835 2835
query25 564 344 337 337
query26 1540 143 142 142
query27 2901 285 282 282
query28 6984 1956 1958 1956
query29 1744 389 395 389
query30 285 144 148 144
query31 972 755 774 755
query32 84 60 57 57
query33 735 280 280 280
query34 865 456 454 454
query35 896 813 770 770
query36 1354 1133 1268 1133
query37 161 70 74 70
query38 3411 3302 3275 3275
query39 1302 1291 1282 1282
query40 292 92 91 91
query41 37 35 35 35
query42 96 92 96 92
query43 528 494 535 494
query44 1092 717 736 717
query45 199 184 185 184
query46 1086 641 632 632
query47 1703 1486 1510 1486
query48 342 260 267 260
query49 1068 332 324 324
query50 791 342 343 342
query51 5405 5268 5306 5268
query52 91 85 91 85
query53 221 152 154 152
query54 1253 560 578 560
query55 99 89 91 89
query56 208 199 197 197
query57 997 945 965 945
query58 228 194 197 194
query59 2877 2736 2693 2693
query60 252 247 234 234
query61 86 81 82 81
query62 616 459 456 456
query63 168 151 154 151
query64 5494 1733 1725 1725
query65 3330 3268 3242 3242
query66 1270 337 333 333
query67 15561 15326 15072 15072
query68 11709 527 535 527
query69 513 249 257 249
query70 1681 1549 1457 1457
query71 480 229 221 221
query72 5633 3558 3516 3516
query73 2629 317 315 315
query74 7052 6480 6421 6421
query75 5018 2299 2265 2265
query76 6266 1186 1122 1122
query77 662 274 255 255
query78 9168 8911 8726 8726
query79 1033 516 515 515
query80 558 365 364 364
query81 454 207 207 207
query82 195 100 101 100
query83 163 131 132 131
query84 245 52 51 51
query85 984 278 268 268
query86 385 353 376 353
query87 3599 3384 3418 3384
query88 2988 2284 2300 2284
query89 336 267 253 253
query90 1927 207 221 207
query91 116 89 96 89
query92 62 51 55 51
query93 1965 449 439 439
query94 898 188 196 188
query95 472 418 411 411
query96 626 321 320 320
query97 4272 4126 4172 4126
query98 205 196 189 189
query99 1120 808 852 808
Total cold run time: 290088 ms
Total hot run time: 179128 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1867602277
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.85 seconds
stream load tsv: 570 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17184213361 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1874880340
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 5d8e5ed471bb0b5ca7f39bf1ef4dec1a2bc0d0f5, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5394 5205 5210 5205
q2 396 169 149 149
q3 1477 1263 1222 1222
q4 1120 832 829 829
q5 3144 3175 3088 3088
q6 229 144 129 129
q7 997 566 521 521
q8 2161 2297 2231 2231
q9 6681 6718 6655 6655
q10 3233 3162 3155 3155
q11 345 216 222 216
q12 387 239 236 236
q13 4400 3688 3714 3688
q14 256 220 223 220
q15 613 546 537 537
q16 460 429 396 396
q17 1042 529 579 529
q18 7024 6790 6878 6790
q19 1647 1552 1578 1552
q20 578 357 352 352
q21 2895 2518 2495 2495
q22 400 332 337 332
Total cold run time: 44879 ms
Total hot run time: 40527 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5137 5114 5108 5108
q2 339 240 231 231
q3 3389 3308 3306 3306
q4 2148 1967 2027 1967
q5 5966 5961 5935 5935
q6 227 131 124 124
q7 2396 1905 1935 1905
q8 3572 3664 3695 3664
q9 9123 8999 9018 8999
q10 3909 3927 3965 3927
q11 611 486 512 486
q12 809 669 633 633
q13 3879 3225 3208 3208
q14 302 296 276 276
q15 623 544 545 544
q16 572 523 529 523
q17 2032 1795 1827 1795
q18 8851 8403 8338 8338
q19 1745 1681 1692 1681
q20 2255 2010 2009 2009
q21 5714 5312 5234 5234
q22 552 517 462 462
Total cold run time: 64151 ms
Total hot run time: 60355 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1874164998
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.96 seconds
stream load tsv: 581 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.1 seconds inserted 10000000 Rows, about 355K ops/s
storage size: 17183577065 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1880294689
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1880465886
<details>
<summary>TPC-H: <b>Total hot run time: 38362 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ef0614bacf3d50ec61c895bb454c3062391f18de, data reload: false
------ Round 1 ----------------------------------
q1 17726 4920 4893 4893
q2 2024 154 141 141
q3 10587 1133 1174 1133
q4 10465 803 798 798
q5 7806 2992 2959 2959
q6 201 125 122 122
q7 914 512 487 487
q8 9270 1975 1978 1975
q9 6437 6359 6315 6315
q10 8251 2981 3007 2981
q11 423 214 222 214
q12 387 222 228 222
q13 18001 3357 3381 3357
q14 232 212 216 212
q15 544 502 510 502
q16 467 387 417 387
q17 939 732 585 585
q18 7230 6620 6711 6620
q19 1542 1506 1509 1506
q20 719 305 322 305
q21 2727 2359 2350 2350
q22 346 299 298 298
Total cold run time: 107238 ms
Total hot run time: 38362 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4918 4881 4868 4868
q2 330 224 238 224
q3 3291 3298 3281 3281
q4 2118 2029 2006 2006
q5 5775 5761 5784 5761
q6 202 116 119 116
q7 2303 1885 1903 1885
q8 3354 3475 3441 3441
q9 8743 8706 8712 8706
q10 3799 3828 3848 3828
q11 552 443 461 443
q12 798 651 628 628
q13 4874 3193 3181 3181
q14 294 290 264 264
q15 539 498 496 496
q16 555 484 514 484
q17 1860 1853 1849 1849
q18 8626 8361 8184 8184
q19 1581 1631 1604 1604
q20 2177 1965 1951 1951
q21 5566 5201 5283 5201
q22 485 455 449 449
Total cold run time: 62740 ms
Total hot run time: 58850 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873730482
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.96 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.2 seconds inserted 10000000 Rows, about 354K ops/s
storage size: 17184273727 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873964143
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 915e6daa9ecdba19b712536737d94890ca5bc618, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5558 5085 5120 5085
q2 389 176 177 176
q3 1464 1214 1189 1189
q4 1095 898 869 869
q5 3126 3068 3092 3068
q6 226 135 140 135
q7 1000 582 499 499
q8 2138 2234 2204 2204
q9 6718 6679 6644 6644
q10 3166 3160 3100 3100
q11 373 212 210 210
q12 381 241 232 232
q13 4357 3644 3637 3637
q14 244 220 225 220
q15 606 563 541 541
q16 447 423 403 403
q17 1038 605 557 557
q18 7057 6708 6774 6708
q19 1625 1538 1466 1466
q20 615 355 341 341
q21 2878 2469 2478 2469
q22 412 323 336 323
Total cold run time: 44913 ms
Total hot run time: 40076 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5110 5108 5051 5051
q2 345 237 246 237
q3 3379 3312 3300 3300
q4 2158 2013 2002 2002
q5 5953 5924 5908 5908
q6 225 124 124 124
q7 2362 1932 1923 1923
q8 3550 3635 3673 3635
q9 9002 9003 8966 8966
q10 3882 3918 3909 3909
q11 579 489 540 489
q12 820 624 667 624
q13 3871 3216 3184 3184
q14 305 285 270 270
q15 616 538 541 538
q16 567 519 526 519
q17 2044 1814 1781 1781
q18 8761 8370 8415 8370
q19 1741 1649 1679 1649
q20 2726 1993 1979 1979
q21 5765 5372 5330 5330
q22 600 501 497 497
Total cold run time: 64361 ms
Total hot run time: 60285 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1878553612
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 1193c42854cc6a5a158f95455b0327d436e44be2, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5469 5116 5125 5116
q2 387 168 149 149
q3 1437 1128 1199 1128
q4 1087 833 778 778
q5 3124 3156 3013 3013
q6 223 132 132 132
q7 993 534 527 527
q8 2156 2255 2291 2255
q9 6694 6678 6666 6666
q10 3186 3112 3154 3112
q11 343 216 203 203
q12 375 231 223 223
q13 4372 3616 3632 3616
q14 250 215 217 215
q15 581 537 531 531
q16 447 398 408 398
q17 1046 554 543 543
q18 7084 6765 6783 6765
q19 1649 1460 1408 1408
q20 604 397 318 318
q21 2896 2511 2504 2504
q22 398 330 311 311
Total cold run time: 44801 ms
Total hot run time: 39911 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5152 5043 4999 4999
q2 338 233 235 233
q3 3354 3340 3306 3306
q4 2152 2028 2076 2028
q5 5951 5914 5926 5914
q6 227 124 128 124
q7 2387 1953 1951 1951
q8 3543 3648 3682 3648
q9 9045 9003 9039 9003
q10 3866 3905 3908 3905
q11 579 470 485 470
q12 809 663 637 637
q13 3863 3172 3180 3172
q14 298 260 271 260
q15 588 532 520 520
q16 540 503 493 493
q17 2040 1831 1797 1797
q18 8802 8332 8420 8332
q19 1740 1652 1701 1652
q20 2271 1998 1956 1956
q21 5739 5361 5386 5361
q22 569 510 514 510
Total cold run time: 63853 ms
Total hot run time: 60271 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1868874744
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.9 seconds
stream load tsv: 564 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 41.9 seconds inserted 10000000 Rows, about 238K ops/s
storage size: 17183788720 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1870241631
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Tpch sf100 test result on commit 09b5c11c66c22757ba3ae439ee053d84ad3a0a8f, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4971 4621 4665 4621
q2 362 161 163 161
q3 1454 1297 1254 1254
q4 1125 940 900 900
q5 3166 3148 3149 3148
q6 248 124 125 124
q7 1027 488 504 488
q8 2242 2249 2245 2245
q9 6687 6732 6678 6678
q10 3197 3248 3260 3248
q11 327 207 217 207
q12 350 211 209 209
q13 4150 3436 3444 3436
q14 246 210 213 210
q15 564 524 533 524
q16 439 382 385 382
q17 1037 787 621 621
q18 7060 6897 6820 6820
q19 1615 1627 1632 1627
q20 538 369 291 291
q21 3140 2637 2714 2637
q22 367 314 314 314
Total cold run time: 44312 ms
Total hot run time: 40145 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4555 4575 4567 4567
q2 269 160 166 160
q3 3393 3369 3353 3353
q4 2227 2201 2205 2201
q5 5721 5721 5710 5710
q6 239 118 118 118
q7 2367 1874 1859 1859
q8 3598 3602 3602 3602
q9 9048 8973 8954 8954
q10 3791 3879 3881 3879
q11 494 356 363 356
q12 780 601 587 587
q13 3895 3168 3211 3168
q14 281 258 262 258
q15 573 513 521 513
q16 498 440 465 440
q17 1974 1959 1952 1952
q18 8644 8174 8136 8136
q19 1735 1755 1758 1755
q20 2258 1929 1948 1929
q21 6121 5769 5756 5756
q22 540 461 470 461
Total cold run time: 63001 ms
Total hot run time: 59714 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873715043
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 6c48386313ed98690be76c3fee38f9fc6e5f2cc5, data reload: false
------ Round 1 ----------------------------------
q1 17601 5268 5163 5163
q2 2051 156 157 156
q3 10581 1140 1140 1140
q4 10206 820 841 820
q5 7813 2952 2889 2889
q6 214 136 133 133
q7 930 508 552 508
q8 9255 2037 2047 2037
q9 6939 6436 6441 6436
q10 8314 3084 3007 3007
q11 436 207 209 207
q12 390 238 237 237
q13 18010 3640 3666 3640
q14 240 223 214 214
q15 579 552 550 550
q16 499 406 400 400
q17 993 479 508 479
q18 7319 6684 6689 6684
q19 1616 1337 1332 1332
q20 751 346 364 346
q21 2859 2361 2459 2361
q22 379 330 325 325
Total cold run time: 107975 ms
Total hot run time: 39064 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5131 5136 5063 5063
q2 337 245 256 245
q3 3323 3313 3289 3289
q4 2140 2028 1997 1997
q5 5845 5780 5807 5780
q6 218 124 123 123
q7 2309 1903 1934 1903
q8 3388 3472 3474 3472
q9 8897 8798 8782 8782
q10 3815 3873 3852 3852
q11 583 476 498 476
q12 802 642 669 642
q13 8969 3209 3208 3208
q14 291 269 270 269
q15 590 534 533 533
q16 551 491 501 491
q17 1951 1754 1762 1754
q18 8812 8405 8339 8339
q19 1628 1610 1630 1610
q20 2218 1964 1967 1964
q21 5575 5388 5282 5282
q22 537 527 464 464
Total cold run time: 67910 ms
Total hot run time: 59538 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873940676
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1868241922
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Tpch sf100 test result on commit 2086add8ec732822b5c32ac2344748500a36f784, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4685 4428 4419 4419
q2 373 146 158 146
q3 1460 1266 1237 1237
q4 1111 886 903 886
q5 3168 3133 3177 3133
q6 247 129 129 129
q7 986 486 492 486
q8 2171 2225 2185 2185
q9 6677 6667 6667 6667
q10 3226 3254 3266 3254
q11 308 186 192 186
q12 348 212 206 206
q13 4535 3809 3796 3796
q14 244 210 211 210
q15 561 521 521 521
q16 436 384 381 381
q17 1002 620 526 526
q18 7035 6758 6850 6758
q19 1532 1439 1407 1407
q20 524 319 314 314
q21 3075 2642 2631 2631
q22 348 280 283 280
Total cold run time: 44052 ms
Total hot run time: 39758 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4332 4392 4328 4328
q2 269 165 170 165
q3 3502 3499 3496 3496
q4 2393 2357 2368 2357
q5 5694 5737 5711 5711
q6 239 122 123 122
q7 2356 1872 1877 1872
q8 3518 3515 3507 3507
q9 9034 8990 8977 8977
q10 3915 4003 3997 3997
q11 488 373 368 368
q12 761 579 599 579
q13 4266 3581 3576 3576
q14 288 250 255 250
q15 581 520 519 519
q16 503 466 447 447
q17 1888 1868 1849 1849
q18 8499 8135 8150 8135
q19 1738 1759 1756 1756
q20 2269 1937 1942 1937
q21 6490 6144 6131 6131
q22 499 414 407 407
Total cold run time: 63522 ms
Total hot run time: 60486 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873970391
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools
```
TPC-DS sf100 test result on commit 915e6daa9ecdba19b712536737d94890ca5bc618, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 943 365 341 341
query2 6414 1873 1932 1873
query3 6650 210 206 206
query4 26654 22652 22455 22455
query5 4716 522 527 522
query6 276 181 182 181
query7 4573 278 266 266
query8 237 210 201 201
query9 8263 2686 2723 2686
query10 432 261 280 261
query11 16145 15474 15528 15474
query12 140 81 75 75
query13 1634 337 343 337
query14 11883 7023 7124 7023
query15 236 187 194 187
query16 6492 289 275 275
query17 1801 499 497 497
query18 1931 274 268 268
query19 252 152 140 140
query20 85 77 79 77
query21 179 103 101 101
query22 4703 4567 4607 4567
query23 32222 31177 31341 31177
query24 11899 2866 2856 2856
query25 574 344 352 344
query26 1757 155 148 148
query27 2864 288 285 285
query28 7011 1960 1949 1949
query29 2055 390 410 390
query30 290 155 147 147
query31 990 757 782 757
query32 83 60 58 58
query33 718 279 276 276
query34 879 453 459 453
query35 929 768 759 759
query36 1350 1222 1203 1203
query37 191 75 73 73
query38 3386 3250 3290 3250
query39 1335 1308 1297 1297
query40 303 97 94 94
query41 37 36 35 35
query42 100 101 90 90
query43 517 497 512 497
query44 1099 707 730 707
query45 200 196 190 190
query46 1064 657 648 648
query47 1638 1624 1569 1569
query48 337 262 264 262
query49 1194 337 332 332
query50 738 391 344 344
query51 5310 5262 5358 5262
query52 97 87 92 87
query53 216 146 148 146
query54 1399 562 604 562
query55 101 90 88 88
query56 217 209 202 202
query57 1035 988 961 961
query58 227 206 207 206
query59 2855 2702 2606 2606
query60 274 254 252 252
query61 86 85 82 82
query62 644 449 447 447
query63 179 140 160 140
query64 5891 1744 1741 1741
query65 3376 3278 3262 3262
query66 1349 338 327 327
query67 16057 15336 15120 15120
query68 12473 526 522 522
query69 545 258 258 258
query70 1753 1505 1510 1505
query71 491 232 231 231
query72 5581 3471 3502 3471
query73 2869 313 311 311
query74 7057 6418 6485 6418
query75 5055 2237 2282 2237
query76 6278 1137 1123 1123
query77 664 258 282 258
query78 9146 8869 8714 8714
query79 1047 519 510 510
query80 551 372 339 339
query81 447 208 208 208
query82 212 102 99 99
query83 183 137 134 134
query84 240 54 54 54
query85 971 281 269 269
query86 391 389 384 384
query87 3503 3392 3318 3318
query88 3008 2268 2283 2268
query89 349 254 258 254
query90 1953 196 203 196
query91 122 90 94 90
query92 58 55 53 53
query93 1454 439 438 438
query94 929 192 188 188
query95 465 422 423 422
query96 622 317 319 317
query97 4246 4187 4164 4164
query98 219 202 192 192
query99 1107 846 835 835
Total cold run time: 293701 ms
Total hot run time: 179474 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873959365
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 915e6daa9ecdba19b712536737d94890ca5bc618, data reload: false
------ Round 1 ----------------------------------
q1 18384 5337 5154 5154
q2 2028 155 145 145
q3 10629 1095 1059 1059
q4 10271 767 846 767
q5 7821 2895 2971 2895
q6 212 135 136 135
q7 926 516 553 516
q8 9277 2023 1977 1977
q9 6807 6458 6329 6329
q10 8244 3097 3012 3012
q11 423 208 210 208
q12 394 236 230 230
q13 18009 3628 3631 3628
q14 241 221 216 216
q15 583 541 529 529
q16 462 391 382 382
q17 974 497 564 497
q18 7444 6756 6701 6701
q19 1591 1383 1381 1381
q20 730 341 352 341
q21 2799 2315 2346 2315
q22 381 333 336 333
Total cold run time: 108630 ms
Total hot run time: 38750 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5068 5094 5076 5076
q2 341 243 259 243
q3 3341 3289 3276 3276
q4 2120 1998 2000 1998
q5 5845 5781 5810 5781
q6 214 124 123 123
q7 2332 1904 1907 1904
q8 3364 3443 3470 3443
q9 8869 8817 8759 8759
q10 3812 3837 3856 3837
q11 594 475 472 472
q12 835 645 630 630
q13 6699 3223 3187 3187
q14 291 286 267 267
q15 609 531 529 529
q16 542 489 522 489
q17 1938 1774 1760 1760
q18 8688 8421 8324 8324
q19 1619 1604 1618 1604
q20 2213 1987 1962 1962
q21 5592 5282 5261 5261
q22 558 484 464 464
Total cold run time: 65484 ms
Total hot run time: 59389 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1878460949
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #28890:
URL: https://github.com/apache/doris/pull/28890#discussion_r1442659883
##########
fe/fe-core/src/main/java/org/apache/doris/plugin/DialectConverterPlugin.java:
##########
@@ -0,0 +1,55 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.plugin;
+
+import org.apache.doris.analysis.StatementBase;
+import org.apache.doris.nereids.parser.Dialect;
+import org.apache.doris.qe.SessionVariable;
+
+import com.google.common.collect.ImmutableSet;
+
+import java.util.List;
+import javax.annotation.Nullable;
+
+/**
+ * Sql dialect adapter interface.
+ */
+public interface DialectConverterPlugin {
+
+ /**
+ * Set dialect set which this plugin can handle
+ */
+ ImmutableSet<Dialect> acceptDialects();
+
+ /**
+ * Override this method if you want to convert sql before parse, returns null as default.
+ * */
+ default @Nullable String convertSql(String originSql, SessionVariable sessionVariable) {
+ return null;
Review Comment:
how about:
```suggestion
return originSql;
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/Dialect.java:
##########
@@ -0,0 +1,117 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.parser;
+
+import javax.annotation.Nullable;
+
+/**
+ * ParseDialect enum, maybe support other dialect.
+ */
+public enum Dialect {
+ /**
+ * Doris parser dialect
+ */
+ DORIS("doris"),
+ /**
+ * Trino parser dialect
+ */
+ TRINO("trino"),
+ /**
+ * Presto parser dialect
+ */
+ PRESTO("presto"),
+ /**
+ * Spark sql parser dialect
+ */
+ SPARK_SQL("spark_sql"),
+ /**
+ * Hive parser dialect
+ */
+ HIVE("hive"),
+ /**
+ * Iceberg parser dialect
+ */
+ ICEBERG("iceberg"),
Review Comment:
iceberg\paimon\hudi\dlf are not query engine, so they don't have sql dialect, remove them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1878496827
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools
```
TPC-DS sf100 test result on commit 1193c42854cc6a5a158f95455b0327d436e44be2, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 930 346 329 329
query2 4345 1950 1842 1842
query3 6644 202 197 197
query4 26064 22381 22384 22381
query5 3249 488 500 488
query6 261 179 172 172
query7 4574 284 266 266
query8 235 194 195 194
query9 8250 2574 2637 2574
query10 483 241 235 235
query11 16110 15570 15513 15513
query12 121 77 74 74
query13 1623 340 324 324
query14 10568 7158 7054 7054
query15 219 180 203 180
query16 6164 268 260 260
query17 1873 489 489 489
query18 1861 278 266 266
query19 197 128 130 128
query20 79 71 72 71
query21 168 94 95 94
query22 4770 4867 4475 4475
query23 32102 31195 31042 31042
query24 12096 2826 2813 2813
query25 552 338 330 330
query26 1461 134 138 134
query27 2922 274 286 274
query28 6940 1958 1945 1945
query29 1487 393 386 386
query30 275 140 148 140
query31 965 775 763 763
query32 82 57 56 56
query33 747 243 238 238
query34 935 436 431 431
query35 875 770 794 770
query36 1336 1176 1194 1176
query37 153 63 67 63
query38 3352 3247 3274 3247
query39 1334 1284 1269 1269
query40 276 85 91 85
query41 37 37 35 35
query42 94 96 80 80
query43 538 475 462 462
query44 1083 710 708 708
query45 191 188 180 180
query46 1052 621 642 621
query47 1665 1556 1549 1549
query48 327 257 259 257
query49 1117 322 323 322
query50 732 361 365 361
query51 5348 5321 5270 5270
query52 87 88 78 78
query53 208 140 138 138
query54 1242 568 568 568
query55 100 82 84 82
query56 197 182 189 182
query57 1021 959 926 926
query58 216 195 203 195
query59 2851 2677 2570 2570
query60 227 207 219 207
query61 84 82 83 82
query62 638 450 455 450
query63 164 144 140 140
query64 5275 1745 1663 1663
query65 3318 3235 3239 3235
query66 1333 332 337 332
query67 15830 15305 15185 15185
query68 10661 526 526 526
query69 524 247 240 240
query70 1747 1510 1448 1448
query71 473 199 219 199
query72 5617 3513 3538 3513
query73 2117 310 315 310
query74 6995 6492 6476 6476
query75 5061 2278 2293 2278
query76 6146 1080 1121 1080
query77 672 266 251 251
query78 9089 8849 8531 8531
query79 1020 511 492 492
query80 531 344 348 344
query81 464 216 206 206
query82 202 91 92 91
query83 163 134 132 132
query84 240 52 53 52
query85 906 285 264 264
query86 391 394 387 387
query87 3593 3378 3346 3346
query88 3074 2247 2244 2244
query89 324 259 261 259
query90 1970 195 189 189
query91 116 89 88 88
query92 63 56 53 53
query93 1404 506 439 439
query94 836 176 177 176
query95 455 413 408 408
query96 608 316 319 316
query97 4277 4141 4192 4141
query98 195 182 191 182
query99 1086 835 773 773
Total cold run time: 282673 ms
Total hot run time: 178080 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1870162012
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1868873378
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Tpch sf100 test result on commit 5bc20737c07089850a44cc55cb45d83d54f35893, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4692 4434 4413 4413
q2 368 140 158 140
q3 1465 1263 1245 1245
q4 1108 941 880 880
q5 3138 3138 3152 3138
q6 248 131 130 130
q7 999 492 489 489
q8 2174 2217 2193 2193
q9 6674 6660 6647 6647
q10 3193 3255 3305 3255
q11 308 188 183 183
q12 379 213 215 213
q13 4558 3804 3793 3793
q14 242 209 218 209
q15 576 518 524 518
q16 434 385 396 385
q17 1030 666 561 561
q18 7175 6783 6699 6699
q19 1529 1424 1427 1424
q20 513 320 299 299
q21 3053 2658 2622 2622
q22 348 282 289 282
Total cold run time: 44204 ms
Total hot run time: 39718 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4344 4317 4326 4317
q2 264 164 170 164
q3 3499 3491 3490 3490
q4 2403 2360 2373 2360
q5 5722 5711 5711 5711
q6 239 125 124 124
q7 2345 1835 1860 1835
q8 3506 3514 3518 3514
q9 8981 8925 8959 8925
q10 3907 3979 4017 3979
q11 478 370 382 370
q12 783 597 610 597
q13 4282 3550 3567 3550
q14 300 250 266 250
q15 578 530 523 523
q16 485 459 463 459
q17 1864 1862 1845 1845
q18 8487 7972 8097 7972
q19 1735 1740 1763 1740
q20 2253 1960 1920 1920
q21 6502 6156 6121 6121
q22 505 448 424 424
Total cold run time: 63462 ms
Total hot run time: 60190 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1868235259
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873700616
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873975064
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.37 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.1 seconds inserted 10000000 Rows, about 355K ops/s
storage size: 17184092262 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1878498909
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.06 seconds
stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 27.0 seconds inserted 10000000 Rows, about 370K ops/s
storage size: 17183808089 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873690412
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873723299
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools
```
TPC-DS sf100 test result on commit 6c48386313ed98690be76c3fee38f9fc6e5f2cc5, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 882 368 351 351
query2 5190 1789 1995 1789
query3 5437 206 203 203
query4 26893 22490 22352 22352
query5 3345 519 523 519
query6 254 185 186 185
query7 4239 278 284 278
query8 231 208 215 208
query9 7325 2766 2588 2588
query10 356 266 237 237
query11 16155 15541 15598 15541
query12 128 78 77 77
query13 1459 325 326 325
query14 10960 7161 7236 7161
query15 228 191 194 191
query16 5971 269 261 261
query17 1741 503 491 491
query18 1848 278 260 260
query19 264 139 135 135
query20 81 82 80 80
query21 177 94 93 93
query22 5077 4588 4886 4588
query23 32235 31354 31286 31286
query24 12307 2798 2859 2798
query25 574 346 344 344
query26 1599 142 141 141
query27 2878 287 280 280
query28 6848 1969 1953 1953
query29 1887 396 376 376
query30 279 149 145 145
query31 1033 757 769 757
query32 87 59 56 56
query33 697 273 291 273
query34 837 457 440 440
query35 867 744 768 744
query36 1400 1214 1281 1214
query37 158 72 74 72
query38 3456 3323 3289 3289
query39 1313 1291 1273 1273
query40 292 88 93 88
query41 38 35 35 35
query42 99 94 94 94
query43 523 491 504 491
query44 1059 716 713 713
query45 196 189 179 179
query46 1085 642 640 640
query47 1670 1679 1596 1596
query48 334 271 265 265
query49 1193 323 326 323
query50 735 318 353 318
query51 5311 5427 5194 5194
query52 107 87 90 87
query53 219 153 143 143
query54 1786 564 583 564
query55 96 90 85 85
query56 214 199 203 199
query57 1019 941 976 941
query58 223 204 206 204
query59 2827 2600 2589 2589
query60 254 245 253 245
query61 86 83 83 83
query62 657 424 482 424
query63 169 143 159 143
query64 5602 1739 1740 1739
query65 3369 3276 3264 3264
query66 1391 338 339 338
query67 15973 15629 15022 15022
query68 12834 536 525 525
query69 502 251 259 251
query70 2246 1551 1445 1445
query71 466 227 222 222
query72 5592 3515 3506 3506
query73 3441 318 313 313
query74 7042 6468 6390 6390
query75 5405 2310 2259 2259
query76 6251 1081 1131 1081
query77 690 284 272 272
query78 9171 8805 8826 8805
query79 1997 502 520 502
query80 540 369 363 363
query81 470 211 211 211
query82 199 99 101 99
query83 181 140 138 138
query84 245 52 55 52
query85 946 285 264 264
query86 380 381 381 381
query87 3508 3365 3342 3342
query88 2948 2251 2257 2251
query89 359 260 258 258
query90 1971 207 207 207
query91 116 88 89 88
query92 62 53 48 48
query93 1616 446 499 446
query94 873 182 185 182
query95 456 412 409 409
query96 620 316 319 316
query97 4307 4224 4183 4183
query98 214 196 191 191
query99 1126 868 806 806
Total cold run time: 289997 ms
Total hot run time: 179166 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1874875590
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 5d8e5ed471bb0b5ca7f39bf1ef4dec1a2bc0d0f5, data reload: false
------ Round 1 ----------------------------------
q1 18791 5172 5083 5083
q2 2027 158 145 145
q3 10610 1080 1111 1080
q4 10213 745 829 745
q5 7813 2925 2836 2836
q6 217 132 135 132
q7 942 525 527 525
q8 9271 2028 2006 2006
q9 6822 6434 6365 6365
q10 8256 3067 3024 3024
q11 414 243 210 210
q12 391 237 236 236
q13 18006 3604 3628 3604
q14 246 212 217 212
q15 593 541 530 530
q16 450 385 394 385
q17 963 524 521 521
q18 7362 6710 6641 6641
q19 1599 1370 1376 1370
q20 689 377 343 343
q21 2814 2363 2471 2363
q22 378 322 324 322
Total cold run time: 108867 ms
Total hot run time: 38678 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5110 5069 5067 5067
q2 351 266 266 266
q3 3344 3312 3272 3272
q4 2099 2029 2015 2015
q5 5810 5806 5786 5786
q6 212 125 123 123
q7 2374 1920 1922 1920
q8 3376 3451 3472 3451
q9 8868 8791 8732 8732
q10 3803 3823 3833 3823
q11 574 479 487 479
q12 840 662 641 641
q13 8065 3218 3234 3218
q14 282 283 270 270
q15 607 526 538 526
q16 566 535 501 501
q17 1972 1786 1741 1741
q18 8718 8307 8277 8277
q19 1631 1591 1602 1591
q20 2177 1956 1961 1956
q21 5517 5270 5191 5191
q22 531 525 495 495
Total cold run time: 66827 ms
Total hot run time: 59341 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1867559915
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #28890:
URL: https://github.com/apache/doris/pull/28890
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1868853553
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1868244056
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.45 seconds
stream load tsv: 564 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17183825534 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1878483269
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 1193c42854cc6a5a158f95455b0327d436e44be2, data reload: false
------ Round 1 ----------------------------------
q1 18110 6148 5113 5113
q2 2029 162 144 144
q3 10602 1127 1095 1095
q4 10243 783 844 783
q5 7816 2896 2718 2718
q6 217 132 132 132
q7 938 507 541 507
q8 9274 1994 2020 1994
q9 6839 6413 6348 6348
q10 8227 3037 3035 3035
q11 435 217 211 211
q12 383 226 225 225
q13 18022 3626 3639 3626
q14 249 214 215 214
q15 553 537 522 522
q16 447 404 384 384
q17 964 542 507 507
q18 7407 6704 6746 6704
q19 1589 1266 1288 1266
q20 725 335 340 335
q21 2793 2414 2437 2414
q22 356 312 331 312
Total cold run time: 108218 ms
Total hot run time: 38589 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5145 5095 5052 5052
q2 336 233 231 231
q3 3338 3279 3280 3279
q4 2146 2040 2034 2034
q5 5815 5770 5795 5770
q6 214 124 124 124
q7 2355 1929 1911 1911
q8 3382 3424 3464 3424
q9 8841 8754 8761 8754
q10 3797 3843 3858 3843
q11 586 476 492 476
q12 804 629 659 629
q13 7348 3222 3217 3217
q14 284 268 269 268
q15 571 524 517 517
q16 559 505 521 505
q17 1921 1798 1763 1763
q18 8709 8394 8315 8315
q19 1631 1555 1574 1555
q20 2186 1949 1935 1935
q21 5692 5251 5217 5217
q22 535 480 473 473
Total cold run time: 66195 ms
Total hot run time: 59292 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1874115671
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1874862998
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1880401564
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.55 seconds
stream load tsv: 579 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 26.3 seconds inserted 10000000 Rows, about 380K ops/s
storage size: 17184003339 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1881119975
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873686979
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1873727779
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 6c48386313ed98690be76c3fee38f9fc6e5f2cc5, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5507 5091 5127 5091
q2 392 164 155 155
q3 1447 1091 1184 1091
q4 1091 818 779 779
q5 3135 3049 3019 3019
q6 223 137 138 137
q7 970 522 553 522
q8 2147 2279 2279 2279
q9 6670 6651 6642 6642
q10 3153 3157 3102 3102
q11 359 228 215 215
q12 376 240 236 236
q13 4401 3650 3652 3650
q14 253 210 220 210
q15 618 562 535 535
q16 470 405 392 392
q17 1030 479 539 479
q18 7056 6774 6831 6774
q19 1640 1512 1499 1499
q20 591 376 321 321
q21 2934 2433 2467 2433
q22 399 334 340 334
Total cold run time: 44862 ms
Total hot run time: 39895 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5130 5053 5117 5053
q2 335 243 249 243
q3 3371 3328 3310 3310
q4 2149 2024 1998 1998
q5 5920 5912 5920 5912
q6 225 127 124 124
q7 2385 1910 1948 1910
q8 3539 3648 3684 3648
q9 9080 9055 8995 8995
q10 3861 3913 3904 3904
q11 587 483 513 483
q12 809 629 637 629
q13 3881 3187 3175 3175
q14 307 271 283 271
q15 597 552 523 523
q16 560 485 529 485
q17 2030 1793 1812 1793
q18 8710 8337 8299 8299
q19 1731 1661 1701 1661
q20 2296 2007 1969 1969
q21 5697 5338 5269 5269
q22 573 478 487 478
Total cold run time: 63773 ms
Total hot run time: 60132 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1874884517
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.41 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17184060281 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1867595840
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Tpch sf100 test result on commit a2ccd8c911de57d0318df6871419b5218b9c5d15, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4681 4405 4455 4405
q2 368 178 158 158
q3 1472 1242 1228 1228
q4 1103 922 921 921
q5 3163 3178 3178 3178
q6 249 131 131 131
q7 1026 488 492 488
q8 2191 2235 2197 2197
q9 6673 6693 6681 6681
q10 3223 3271 3266 3266
q11 307 183 187 183
q12 353 216 207 207
q13 4564 3840 3793 3793
q14 241 219 210 210
q15 567 522 516 516
q16 434 379 388 379
q17 1025 686 571 571
q18 6992 6733 6804 6733
q19 1531 1439 1428 1428
q20 503 339 295 295
q21 3109 2639 2654 2639
q22 346 279 278 278
Total cold run time: 44121 ms
Total hot run time: 39885 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4335 4328 4374 4328
q2 270 167 168 167
q3 3518 3505 3493 3493
q4 2368 2355 2361 2355
q5 5708 5719 5708 5708
q6 241 124 125 124
q7 2398 1903 1865 1865
q8 3514 3531 3522 3522
q9 9053 9013 9001 9001
q10 3906 4006 4007 4006
q11 487 362 358 358
q12 769 592 594 592
q13 4307 3554 3554 3554
q14 288 258 256 256
q15 577 522 523 522
q16 501 461 455 455
q17 1888 1853 1853 1853
q18 8561 7905 8156 7905
q19 1722 1811 1759 1759
q20 2247 1949 1934 1934
q21 6525 6139 6173 6139
q22 506 454 423 423
Total cold run time: 63689 ms
Total hot run time: 60319 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1868850318
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1870196126
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.35 seconds
stream load tsv: 564 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17183848524 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1880477425
<details>
<summary>TPC-DS: <b>Total hot run time: 184493 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ef0614bacf3d50ec61c895bb454c3062391f18de, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 945 342 336 336
query2 6566 2018 1962 1962
query3 6720 230 225 225
query4 26347 22577 22369 22369
query5 5534 598 565 565
query6 339 211 209 209
query7 4651 296 289 289
query8 260 225 206 206
query9 8617 2990 2915 2915
query10 556 348 326 326
query11 16217 15832 15622 15622
query12 168 97 106 97
query13 1694 345 338 338
query14 13525 7612 7546 7546
query15 262 235 219 219
query16 6469 345 347 345
query17 1694 534 508 508
query18 1972 312 306 306
query19 249 159 156 156
query20 107 108 105 105
query21 189 93 93 93
query22 5078 5030 5031 5030
query23 32347 31245 31175 31175
query24 11763 2920 2923 2920
query25 597 371 367 367
query26 1780 172 173 172
query27 2790 313 317 313
query28 7142 2181 2169 2169
query29 2063 415 414 414
query30 299 146 149 146
query31 1016 829 826 826
query32 133 85 90 85
query33 848 350 343 343
query34 858 476 513 476
query35 968 862 874 862
query36 1335 1295 1274 1274
query37 203 88 96 88
query38 3481 3371 3337 3337
query39 1360 1308 1296 1296
query40 307 92 91 91
query41 39 39 36 36
query42 111 107 103 103
query43 570 508 517 508
query44 1298 813 825 813
query45 220 201 198 198
query46 1042 686 718 686
query47 1761 1631 1613 1613
query48 349 279 313 279
query49 1207 311 318 311
query50 720 325 321 321
query51 5532 5320 5356 5320
query52 98 106 92 92
query53 247 164 173 164
query54 1395 658 687 658
query55 109 104 98 98
query56 271 250 257 250
query57 1055 988 989 988
query58 310 286 288 286
query59 3089 3000 2888 2888
query60 312 298 303 298
query61 145 150 145 145
query62 567 463 502 463
query63 200 178 184 178
query64 5915 1813 1763 1763
query65 3448 3356 3349 3349
query66 1392 426 386 386
query67 15513 15404 15602 15404
query68 13054 557 551 551
query69 585 326 333 326
query70 1754 1587 1508 1508
query71 602 294 308 294
query72 5244 3486 3474 3474
query73 2933 341 351 341
query74 7103 6595 6546 6546
query75 5147 2260 2284 2260
query76 6453 1098 1245 1098
query77 753 366 338 338
query78 9149 8599 8593 8593
query79 1083 528 522 522
query80 543 335 333 333
query81 469 212 215 212
query82 231 108 105 105
query83 188 138 140 138
query84 256 58 58 58
query85 970 279 273 273
query86 426 388 428 388
query87 3634 3505 3498 3498
query88 2992 2529 2442 2442
query89 381 278 280 278
query90 1994 246 262 246
query91 123 96 94 94
query92 93 73 80 73
query93 1199 436 488 436
query94 902 243 247 243
query95 546 500 508 500
query96 636 345 337 337
query97 4487 4199 4213 4199
query98 236 222 215 215
query99 1165 857 855 855
Total cold run time: 299695 ms
Total hot run time: 184493 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org