You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zzzxl1993 (via GitHub)" <gi...@apache.org> on 2024/03/21 09:33:53 UTC
[PR] [fix](tokenize) support statements with spaces [doris]
zzzxl1993 opened a new pull request, #32625:
URL: https://github.com/apache/doris/pull/32625
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2011818220
<details>
<summary>TPC-DS: <b>Total hot run time: 180916 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fe2a250d9a359c0c2ef5c2f299d1524a595f377f, data reload: false
query1 917 354 354 354
query2 7446 2013 2027 2013
query3 6710 213 216 213
query4 31166 20519 20730 20519
query5 4413 425 392 392
query6 291 177 172 172
query7 4631 293 280 280
query8 227 169 183 169
query9 9932 2281 2262 2262
query10 461 245 239 239
query11 14791 14079 14151 14079
query12 144 88 87 87
query13 1627 406 419 406
query14 11787 10812 10434 10434
query15 323 199 200 199
query16 8157 265 259 259
query17 1931 577 547 547
query18 2068 297 281 281
query19 264 158 155 155
query20 94 87 85 85
query21 205 133 127 127
query22 4704 4471 4438 4438
query23 31745 31219 31207 31207
query24 10701 2876 2810 2810
query25 615 386 387 386
query26 1179 156 158 156
query27 3044 337 350 337
query28 8008 1904 1862 1862
query29 887 658 629 629
query30 296 148 145 145
query31 971 722 710 710
query32 91 59 60 59
query33 776 259 251 251
query34 1061 477 481 477
query35 834 615 609 609
query36 977 896 862 862
query37 121 81 75 75
query38 3600 3443 3423 3423
query39 1443 1390 1369 1369
query40 219 114 109 109
query41 51 50 46 46
query42 100 94 97 94
query43 489 473 463 463
query44 1287 721 734 721
query45 273 258 263 258
query46 1116 714 686 686
query47 1655 1587 1604 1587
query48 433 348 354 348
query49 1125 346 336 336
query50 753 378 364 364
query51 6697 6635 6599 6599
query52 104 91 96 91
query53 337 271 281 271
query54 305 236 246 236
query55 87 80 82 80
query56 253 237 236 236
query57 1087 1000 1031 1000
query58 245 208 206 206
query59 3051 2646 2674 2646
query60 267 245 255 245
query61 119 117 119 117
query62 619 398 397 397
query63 300 273 279 273
query64 5742 3956 3791 3791
query65 3081 2986 3016 2986
query66 862 341 335 335
query67 14922 14634 14570 14570
query68 8941 522 518 518
query69 636 372 374 372
query70 1207 1166 1155 1155
query71 522 277 283 277
query72 6395 2652 2506 2506
query73 1196 315 325 315
query74 7810 6504 6588 6504
query75 4707 2882 2827 2827
query76 5366 976 859 859
query77 745 263 253 253
query78 10331 9508 9572 9508
query79 13026 513 505 505
query80 2140 388 371 371
query81 517 217 218 217
query82 511 201 198 198
query83 239 147 154 147
query84 287 76 74 74
query85 1246 325 304 304
query86 361 278 301 278
query87 3743 3552 3518 3518
query88 5137 2294 2286 2286
query89 510 363 362 362
query90 2062 176 177 176
query91 181 150 134 134
query92 64 46 45 45
query93 7494 493 478 478
query94 1339 174 173 173
query95 430 330 330 330
query96 617 272 268 268
query97 3063 2847 2878 2847
query98 226 212 233 212
query99 1029 797 731 731
Total cold run time: 316295 ms
Total hot run time: 180916 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2012235493
TeamCity be ut coverage result:
Function Coverage: 35.27% (8730/24751)
Line Coverage: 27.09% (71490/263922)
Region Coverage: 26.31% (37080/140916)
Branch Coverage: 23.23% (18969/81650)
Coverage Report: http://coverage.selectdb-in.cc/coverage/895c58863b28660487255d6f66aadfd41d7bf430_895c58863b28660487255d6f66aadfd41d7bf430/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2012178841
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2011791181
<details>
<summary>TPC-H: <b>Total hot run time: 38576 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fe2a250d9a359c0c2ef5c2f299d1524a595f377f, data reload: false
------ Round 1 ----------------------------------
q1 17673 4443 4184 4184
q2 2041 160 142 142
q3 10748 1206 1215 1206
q4 10572 848 850 848
q5 7511 3033 3034 3033
q6 212 128 127 127
q7 1084 609 608 608
q8 9703 2058 2027 2027
q9 7249 6670 6572 6572
q10 9706 3508 3590 3508
q11 460 233 226 226
q12 417 209 209 209
q13 18452 2889 2850 2850
q14 241 212 214 212
q15 496 462 451 451
q16 462 355 356 355
q17 977 628 579 579
q18 7296 6486 6597 6486
q19 3269 1444 1449 1444
q20 554 253 243 243
q21 3598 2961 2977 2961
q22 365 305 306 305
Total cold run time: 113086 ms
Total hot run time: 38576 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4201 4096 4116 4096
q2 326 223 220 220
q3 2987 2895 2857 2857
q4 1869 1586 1579 1579
q5 5246 5313 5282 5282
q6 193 119 121 119
q7 2230 1842 1890 1842
q8 3193 3342 3321 3321
q9 8618 8626 8596 8596
q10 3714 3797 3734 3734
q11 551 445 445 445
q12 737 578 571 571
q13 16911 2851 2924 2851
q14 287 256 268 256
q15 489 452 453 452
q16 459 405 421 405
q17 1757 1526 1491 1491
q18 7409 7318 7198 7198
q19 1648 1508 1567 1508
q20 1918 1709 1713 1709
q21 4733 4638 4796 4638
q22 516 498 429 429
Total cold run time: 69992 ms
Total hot run time: 53599 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2012260116
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 895c58863b28660487255d6f66aadfd41d7bf430 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select: 21.7 seconds inserted 10000000 Rows, about 460K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 closed pull request #32625: [fix](tokenize) support statements with spaces
URL: https://github.com/apache/doris/pull/32625
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2011755890
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2012253471
<details>
<summary>TPC-DS: <b>Total hot run time: 180574 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 895c58863b28660487255d6f66aadfd41d7bf430, data reload: false
query1 940 361 353 353
query2 7462 1967 2038 1967
query3 6702 218 210 210
query4 31182 20543 20682 20543
query5 4352 474 423 423
query6 280 174 171 171
query7 4643 289 288 288
query8 228 164 161 161
query9 9910 2284 2280 2280
query10 464 253 254 253
query11 17065 14258 14067 14067
query12 132 91 84 84
query13 1626 410 431 410
query14 11413 10503 10345 10345
query15 257 207 192 192
query16 8163 251 254 251
query17 1964 576 545 545
query18 2068 280 289 280
query19 217 150 155 150
query20 88 89 85 85
query21 196 133 123 123
query22 4700 4437 4412 4412
query23 31915 30930 31102 30930
query24 10823 2856 2869 2856
query25 630 388 398 388
query26 1448 154 153 153
query27 3130 349 350 349
query28 8146 1873 1879 1873
query29 891 657 616 616
query30 301 149 146 146
query31 979 730 724 724
query32 95 58 58 58
query33 770 278 253 253
query34 1064 493 485 485
query35 832 609 618 609
query36 988 881 874 874
query37 130 79 79 79
query38 3555 3473 3504 3473
query39 1480 1372 1446 1372
query40 213 109 111 109
query41 52 47 48 47
query42 105 99 97 97
query43 484 467 455 455
query44 1143 709 706 706
query45 275 254 243 243
query46 1116 682 681 681
query47 1674 1605 1600 1600
query48 433 365 356 356
query49 1141 348 340 340
query50 771 380 381 380
query51 6750 6549 6687 6549
query52 97 90 86 86
query53 344 268 275 268
query54 297 234 231 231
query55 83 80 84 80
query56 300 222 219 219
query57 1094 1016 1006 1006
query58 231 199 210 199
query59 2703 2651 2557 2557
query60 265 246 262 246
query61 94 93 97 93
query62 598 409 404 404
query63 302 275 273 273
query64 5598 3960 3744 3744
query65 3111 3047 2997 2997
query66 1436 337 340 337
query67 14819 14628 14319 14319
query68 5163 521 514 514
query69 563 364 375 364
query70 1245 1197 1105 1105
query71 413 288 288 288
query72 6522 2847 2644 2644
query73 719 316 327 316
query74 6928 6540 6634 6540
query75 3546 2831 2860 2831
query76 3536 874 1004 874
query77 613 273 269 269
query78 10341 9510 9583 9510
query79 8194 513 528 513
query80 1795 416 412 412
query81 550 223 225 223
query82 1255 214 209 209
query83 325 148 150 148
query84 293 83 86 83
query85 1813 392 371 371
query86 489 293 305 293
query87 3753 3520 3564 3520
query88 4983 2335 2293 2293
query89 484 371 363 363
query90 1956 181 181 181
query91 181 134 136 134
query92 66 46 46 46
query93 5743 495 483 483
query94 1251 185 176 176
query95 437 328 342 328
query96 601 268 266 266
query97 3088 2872 2856 2856
query98 217 211 214 211
query99 1145 757 789 757
Total cold run time: 304966 ms
Total hot run time: 180574 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2011746353
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2011753815
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2012164362
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2011746573
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2011786585
TeamCity be ut coverage result:
Function Coverage: 35.27% (8730/24749)
Line Coverage: 27.09% (71485/263882)
Region Coverage: 26.32% (37073/140872)
Branch Coverage: 23.23% (18958/81620)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fe2a250d9a359c0c2ef5c2f299d1524a595f377f_fe2a250d9a359c0c2ef5c2f299d1524a595f377f/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2011761440
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](tokenize) support statements with spaces [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32625:
URL: https://github.com/apache/doris/pull/32625#issuecomment-2012220087
<details>
<summary>TPC-H: <b>Total hot run time: 37951 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 895c58863b28660487255d6f66aadfd41d7bf430, data reload: false
------ Round 1 ----------------------------------
q1 17654 4362 4153 4153
q2 2036 143 143 143
q3 10705 1134 1668 1134
q4 10411 835 778 778
q5 7489 2971 2985 2971
q6 205 125 124 124
q7 1038 592 560 560
q8 9360 2094 2032 2032
q9 6980 6559 6479 6479
q10 8379 3371 3524 3371
q11 424 224 211 211
q12 375 205 202 202
q13 17798 2855 2844 2844
q14 228 195 201 195
q15 497 471 456 456
q16 459 352 357 352
q17 970 653 573 573
q18 7369 6664 6509 6509
q19 4439 1478 1495 1478
q20 543 245 245 245
q21 3610 2935 2849 2849
q22 345 330 292 292
Total cold run time: 111314 ms
Total hot run time: 37951 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4172 4121 4123 4121
q2 318 232 226 226
q3 2982 2828 2863 2828
q4 1863 1560 1601 1560
q5 5295 5267 5316 5267
q6 196 116 118 116
q7 2246 1832 1826 1826
q8 3186 3308 3328 3308
q9 8664 8592 8611 8592
q10 3741 3716 3678 3678
q11 550 436 434 434
q12 711 561 587 561
q13 16902 2915 2863 2863
q14 281 266 263 263
q15 482 449 426 426
q16 471 399 424 399
q17 1751 1513 1483 1483
q18 7571 7222 7160 7160
q19 1613 1572 1557 1557
q20 1910 1727 1716 1716
q21 4946 4737 4751 4737
q22 556 459 454 454
Total cold run time: 70407 ms
Total hot run time: 53575 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org