You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "kaka11chen (via GitHub)" <gi...@apache.org> on 2024/01/04 17:03:36 UTC
[PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
kaka11chen opened a new pull request, #29562:
URL: https://github.com/apache/doris/pull/29562
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1882136599
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1880124333
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1880501085
<details>
<summary>TPC-H: <b>Total hot run time: 38650 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 512f52aaf0f1d16b374420aa4e98f2e51b28c0c8, data reload: false
------ Round 1 ----------------------------------
q1 17708 5021 4960 4960
q2 2025 155 144 144
q3 10583 1102 1130 1102
q4 10197 821 825 821
q5 7772 3024 2957 2957
q6 203 123 123 123
q7 910 507 493 493
q8 9273 2011 1995 1995
q9 6537 6464 6387 6387
q10 8243 3037 3050 3037
q11 434 223 222 222
q12 379 211 213 211
q13 17991 3395 3363 3363
q14 238 213 209 209
q15 554 512 502 502
q16 454 388 397 388
q17 958 708 597 597
q18 7208 6699 6657 6657
q19 1566 1525 1523 1523
q20 740 320 297 297
q21 2766 2396 2350 2350
q22 361 312 319 312
Total cold run time: 107100 ms
Total hot run time: 38650 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4903 4963 4915 4915
q2 342 231 248 231
q3 3293 3291 3277 3277
q4 2115 2009 2025 2009
q5 5834 5924 6022 5924
q6 205 119 119 119
q7 2435 2021 2019 2019
q8 3423 3449 3485 3449
q9 8796 8744 8732 8732
q10 3787 3845 3840 3840
q11 562 428 442 428
q12 796 630 622 622
q13 4000 3209 3177 3177
q14 288 284 263 263
q15 543 505 506 505
q16 544 497 477 477
q17 1875 1872 1882 1872
q18 8602 8333 8287 8287
q19 1593 1651 1638 1638
q20 2204 1942 1936 1936
q21 5563 5294 5208 5208
q22 509 457 466 457
Total cold run time: 62212 ms
Total hot run time: 59385 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1877566626
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools
```
TPC-DS sf100 test result on commit 30bf19776347bb757919f0776e2053b51cb2b283, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 915 348 328 328
query2 6436 1925 1952 1925
query3 6642 203 199 199
query4 26458 22506 22346 22346
query5 3883 560 504 504
query6 266 185 181 181
query7 4580 284 268 268
query8 225 196 194 194
query9 8269 2565 2576 2565
query10 436 244 250 244
query11 16229 15561 15476 15476
query12 136 79 76 76
query13 1628 333 341 333
query14 12068 7170 7094 7094
query15 231 181 188 181
query16 6405 272 266 266
query17 1862 489 485 485
query18 1973 270 266 266
query19 195 140 136 136
query20 79 73 76 73
query21 189 99 95 95
query22 4816 4632 4712 4632
query23 32135 31159 31233 31159
query24 11867 2853 2854 2853
query25 600 343 331 331
query26 1769 143 140 140
query27 2877 276 278 276
query28 6996 1927 1912 1912
query29 2080 386 396 386
query30 294 139 143 139
query31 988 751 773 751
query32 93 58 55 55
query33 746 262 265 262
query34 937 452 440 440
query35 872 737 789 737
query36 1309 1119 1177 1119
query37 186 65 66 65
query38 3432 3304 3264 3264
query39 1331 1295 1273 1273
query40 301 90 85 85
query41 40 35 35 35
query42 89 91 80 80
query43 540 514 468 468
query44 1062 698 706 698
query45 195 189 178 178
query46 1072 661 641 641
query47 1615 1614 1513 1513
query48 327 267 253 253
query49 1222 307 310 307
query50 726 342 375 342
query51 5325 5360 5340 5340
query52 100 85 84 84
query53 216 140 141 140
query54 1376 550 558 550
query55 104 88 86 86
query56 195 168 186 168
query57 1001 957 905 905
query58 218 205 192 192
query59 2805 2681 2729 2681
query60 230 220 237 220
query61 88 86 86 86
query62 627 463 462 462
query63 163 141 146 141
query64 5835 1761 1736 1736
query65 3351 3254 3280 3254
query66 1397 333 338 333
query67 15743 15842 15183 15183
query68 12621 526 550 526
query69 528 255 241 241
query70 1715 1503 1490 1490
query71 498 219 199 199
query72 5716 3580 3616 3580
query73 3017 318 306 306
query74 6977 6374 6429 6374
query75 5198 2284 2274 2274
query76 6261 1084 1131 1084
query77 650 243 252 243
query78 9094 8635 8630 8630
query79 1033 512 512 512
query80 545 350 344 344
query81 466 207 205 205
query82 204 94 98 94
query83 157 139 138 138
query84 241 54 52 52
query85 954 310 273 273
query86 388 356 386 356
query87 3581 3341 3361 3341
query88 3014 2188 2180 2180
query89 334 270 263 263
query90 1932 191 185 185
query91 117 92 92 92
query92 60 56 51 51
query93 1567 442 491 442
query94 803 175 177 175
query95 455 411 404 404
query96 609 326 309 309
query97 4323 4189 4165 4165
query98 216 199 190 190
query99 1121 837 787 787
Total cold run time: 292791 ms
Total hot run time: 178695 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1880305224
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1877460441
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1880142164
<details>
<summary>TPC-H: <b>Total hot run time: 38404 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4f70df81668390f7d05d1ba594be26438d35190e, data reload: false
------ Round 1 ----------------------------------
q1 17671 4994 4925 4925
q2 2038 155 145 145
q3 10578 1120 1144 1120
q4 10206 742 825 742
q5 7791 2940 2856 2856
q6 206 121 123 121
q7 926 513 493 493
q8 9317 2034 2015 2015
q9 6454 6434 6331 6331
q10 8235 3088 3097 3088
q11 444 224 224 224
q12 383 213 210 210
q13 18021 3394 3361 3361
q14 237 214 210 210
q15 545 513 515 513
q16 441 394 384 384
q17 962 676 564 564
q18 7386 6742 6662 6662
q19 1565 1509 1526 1509
q20 754 315 301 301
q21 2757 2406 2319 2319
q22 357 315 311 311
Total cold run time: 107274 ms
Total hot run time: 38404 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4913 4923 4848 4848
q2 349 238 247 238
q3 3313 3292 3275 3275
q4 2107 2036 2046 2036
q5 5799 5834 5817 5817
q6 203 115 114 114
q7 2340 1927 1906 1906
q8 3401 3497 3486 3486
q9 8862 8759 8777 8759
q10 3800 3855 3841 3841
q11 560 452 455 452
q12 811 627 630 627
q13 7485 3202 3206 3202
q14 281 255 251 251
q15 544 493 499 493
q16 564 494 485 485
q17 1881 1872 1873 1872
q18 8673 8463 8267 8267
q19 1614 1637 1632 1632
q20 2182 1980 1967 1967
q21 5546 5280 5304 5280
q22 498 449 464 449
Total cold run time: 65726 ms
Total hot run time: 59297 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1881503521
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1881540242
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1877551889
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 30bf19776347bb757919f0776e2053b51cb2b283, data reload: false
------ Round 1 ----------------------------------
q1 17675 5170 5155 5155
q2 2016 163 142 142
q3 10537 1135 1170 1135
q4 10185 800 869 800
q5 7796 2991 2818 2818
q6 218 133 133 133
q7 932 506 535 506
q8 9278 2035 2039 2035
q9 6860 6469 6411 6411
q10 8205 3031 2994 2994
q11 444 213 209 209
q12 385 217 219 217
q13 18016 3657 3639 3639
q14 253 220 221 220
q15 565 521 517 517
q16 459 416 395 395
q17 967 480 493 480
q18 7386 6740 6697 6697
q19 1582 1422 1333 1333
q20 756 363 306 306
q21 2792 2380 2412 2380
q22 361 317 321 317
Total cold run time: 107668 ms
Total hot run time: 38839 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5117 5064 5107 5064
q2 338 247 238 238
q3 3336 3308 3283 3283
q4 2135 1976 1982 1976
q5 5832 5813 5796 5796
q6 215 125 127 125
q7 2308 1926 1918 1918
q8 3362 3479 3471 3471
q9 8844 8809 8768 8768
q10 3813 3853 3883 3853
q11 606 466 480 466
q12 789 630 618 618
q13 7432 3231 3216 3216
q14 296 268 263 263
q15 582 525 521 521
q16 575 477 512 477
q17 1976 1767 1761 1761
q18 8734 8396 8360 8360
q19 1615 1556 1591 1556
q20 2191 1958 1948 1948
q21 5592 5283 5258 5258
q22 536 497 483 483
Total cold run time: 66224 ms
Total hot run time: 59419 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #29562:
URL: https://github.com/apache/doris/pull/29562#discussion_r1442047488
##########
be/src/vec/runtime/vdatetime_value.h:
##########
@@ -753,9 +753,11 @@ class DateV2Value {
// Constructor
DateV2Value() : date_v2_value_(0, 0, 0, 0, 0, 0, 0) {}
- DateV2Value(DateV2Value<T>& other) { int_val_ = other.to_date_int_val(); }
+ DateV2Value(underlying_value int_val) : int_val_(int_val) {}
- DateV2Value(const DateV2Value<T>& other) { int_val_ = other.to_date_int_val(); }
+ DateV2Value(DateV2Value<T>& other) : int_val_(other.int_val_) {}
Review Comment:
warning: use '= default' to define a trivial copy constructor [modernize-use-equals-default]
```suggestion
DateV2Value(DateV2Value<T>& other) : int_val_(other.int_val_) = default;
```
##########
be/src/vec/runtime/vdatetime_value.h:
##########
@@ -753,9 +753,11 @@
// Constructor
DateV2Value() : date_v2_value_(0, 0, 0, 0, 0, 0, 0) {}
- DateV2Value(DateV2Value<T>& other) { int_val_ = other.to_date_int_val(); }
+ DateV2Value(underlying_value int_val) : int_val_(int_val) {}
- DateV2Value(const DateV2Value<T>& other) { int_val_ = other.to_date_int_val(); }
+ DateV2Value(DateV2Value<T>& other) : int_val_(other.int_val_) {}
+
+ DateV2Value(const DateV2Value<T>& other) : int_val_(other.int_val_) {}
Review Comment:
warning: use '= default' to define a trivial copy constructor [modernize-use-equals-default]
```suggestion
DateV2Value(const DateV2Value<T>& other) : int_val_(other.int_val_) = default;
```
##########
be/src/vec/runtime/vdatetime_value.h:
##########
@@ -1558,7 +1565,15 @@
static bool get_dict_init();
- DateV2Value<DateV2ValueType> operator[](int day) const;
+ inline DateV2Value<DateV2ValueType> operator[](int day) const {
+ int index = day + DAY_BEFORE_EPOCH;
+ if (LIKELY(index >= 0 && index < DICT_DAYS)) {
Review Comment:
warning: boolean expression can be simplified by DeMorgan's theorem [readability-simplify-boolean-expr]
```cpp
if (LIKELY(index >= 0 && index < DICT_DAYS)) {
^
```
<details>
<summary>Additional context</summary>
**be/src/common/compiler_util.h:34:** expanded from macro 'LIKELY'
```cpp
#define LIKELY(expr) __builtin_expect(!!(expr), 1)
^
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1882497554
run external
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "kaka11chen (via GitHub)" <gi...@apache.org>.
kaka11chen closed pull request #29562: [Opt] (exec) (multi-catalog) Optimize date type reading.
URL: https://github.com/apache/doris/pull/29562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1877570161
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.7 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.7 seconds inserted 10000000 Rows, about 361K ops/s
storage size: 17183857990 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1877554336
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 30bf19776347bb757919f0776e2053b51cb2b283, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5468 5204 5088 5088
q2 384 171 158 158
q3 1462 1143 1150 1143
q4 1061 825 794 794
q5 3124 3019 3080 3019
q6 220 135 142 135
q7 957 531 534 531
q8 2160 2222 2229 2222
q9 6712 6654 6657 6654
q10 3163 3144 3123 3123
q11 359 205 219 205
q12 367 215 218 215
q13 4409 3676 3620 3620
q14 251 222 222 222
q15 576 538 528 528
q16 450 417 384 384
q17 1047 558 551 551
q18 7079 6720 6813 6720
q19 1645 1521 1543 1521
q20 575 345 329 329
q21 2892 2484 2477 2477
q22 380 334 339 334
Total cold run time: 44741 ms
Total hot run time: 39973 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5135 5072 5139 5072
q2 331 235 239 235
q3 3378 3340 3308 3308
q4 2142 2038 2032 2032
q5 5941 5929 5913 5913
q6 223 127 130 127
q7 2356 1912 1928 1912
q8 3557 3688 3648 3648
q9 9054 9007 8960 8960
q10 3867 3879 3876 3876
q11 611 501 463 463
q12 787 628 647 628
q13 3866 3206 3183 3183
q14 308 265 262 262
q15 589 533 525 525
q16 582 503 525 503
q17 2045 1827 1791 1791
q18 8723 8322 8424 8322
q19 1728 1684 1707 1684
q20 2256 1987 1961 1961
q21 5665 5327 5285 5285
q22 568 520 531 520
Total cold run time: 63712 ms
Total hot run time: 60210 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1880144835
<details>
<summary>TPC-DS: <b>Total hot run time: 184601 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4f70df81668390f7d05d1ba594be26438d35190e, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 936 341 335 335
query2 6601 2092 2032 2032
query3 6718 240 232 232
query4 26361 22568 22470 22470
query5 5789 590 615 590
query6 368 220 206 206
query7 4635 299 295 295
query8 259 231 223 223
query9 8805 2932 2968 2932
query10 523 382 343 343
query11 16413 15720 15645 15645
query12 165 106 94 94
query13 1676 355 358 355
query14 13220 7511 7661 7511
query15 290 249 229 229
query16 6455 324 337 324
query17 1718 524 507 507
query18 1971 320 297 297
query19 258 158 157 157
query20 108 104 97 97
query21 188 90 101 90
query22 4962 4923 4832 4832
query23 32401 31257 30983 30983
query24 12051 2927 2913 2913
query25 616 369 362 362
query26 1776 172 168 168
query27 2963 324 306 306
query28 6775 2174 2163 2163
query29 2059 412 407 407
query30 285 143 151 143
query31 1000 803 829 803
query32 129 94 91 91
query33 851 349 354 349
query34 915 500 485 485
query35 1012 915 860 860
query36 1341 1238 1261 1238
query37 213 84 95 84
query38 3487 3389 3354 3354
query39 1354 1306 1283 1283
query40 308 91 87 87
query41 40 39 37 37
query42 109 105 105 105
query43 562 523 542 523
query44 1282 848 824 824
query45 212 212 202 202
query46 1057 683 699 683
query47 1775 1638 1579 1579
query48 378 301 290 290
query49 1209 312 329 312
query50 746 328 368 328
query51 5392 5241 5254 5241
query52 101 103 104 103
query53 234 182 169 169
query54 1424 661 687 661
query55 113 109 104 104
query56 258 256 268 256
query57 1073 1027 984 984
query58 319 269 289 269
query59 3181 2929 2966 2929
query60 328 297 305 297
query61 142 173 154 154
query62 587 473 517 473
query63 208 177 185 177
query64 5943 1781 1731 1731
query65 3431 3358 3348 3348
query66 1380 398 420 398
query67 15699 15719 15631 15631
query68 12373 568 563 563
query69 603 302 325 302
query70 1697 1590 1606 1590
query71 608 258 288 258
query72 5368 3456 3459 3456
query73 2187 350 373 350
query74 7046 6439 6523 6439
query75 4897 2270 2288 2270
query76 6386 1250 1214 1214
query77 774 349 332 332
query78 9060 8650 8702 8650
query79 1039 525 535 525
query80 569 362 363 362
query81 461 211 217 211
query82 239 109 116 109
query83 167 144 144 144
query84 252 57 57 57
query85 918 277 273 273
query86 422 432 400 400
query87 3626 3445 3451 3445
query88 3424 2458 2550 2458
query89 358 297 288 288
query90 1829 267 233 233
query91 124 98 96 96
query92 91 84 87 84
query93 2101 441 484 441
query94 825 231 246 231
query95 585 508 498 498
query96 656 354 346 346
query97 4367 4231 4212 4212
query98 247 238 209 209
query99 1134 872 902 872
Total cold run time: 299571 ms
Total hot run time: 184601 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1880145685
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.26 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.3 seconds inserted 10000000 Rows, about 366K ops/s
storage size: 17183810812 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Opt] (exec) (multi-catalog) Optimize date type reading. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29562:
URL: https://github.com/apache/doris/pull/29562#issuecomment-1880415299
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.47 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.1 seconds inserted 10000000 Rows, about 369K ops/s
storage size: 17183346151 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org