You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "SWJTU-ZhangLei (via GitHub)" <gi...@apache.org> on 2024/03/20 10:15:09 UTC
[PR] [feature](merge-cloud) Add several cloud metrics [doris]
SWJTU-ZhangLei opened a new pull request, #32556:
URL: https://github.com/apache/doris/pull/32556
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2017771645
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit d2c22a6d12f8b88a2adb4b45f5941f5252da8107 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 22.2 seconds inserted 10000000 Rows, about 450K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #32556:
URL: https://github.com/apache/doris/pull/32556
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "SWJTU-ZhangLei (via GitHub)" <gi...@apache.org>.
SWJTU-ZhangLei commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2012089359
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2017149234
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit db8e17a26643208d87ef84b2221608f6baf24b34 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 20.7 seconds inserted 10000000 Rows, about 483K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "SWJTU-ZhangLei (via GitHub)" <gi...@apache.org>.
SWJTU-ZhangLei commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2011658304
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2011752035
<details>
<summary>TPC-DS: <b>Total hot run time: 180165 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d875478d466f8b421e8d9209c18dc5187f337f89, data reload: false
query1 932 358 333 333
query2 7488 2011 2106 2011
query3 6704 212 219 212
query4 31176 20529 20500 20500
query5 4341 388 502 388
query6 268 174 171 171
query7 4625 293 285 285
query8 226 167 165 165
query9 9749 2261 2208 2208
query10 465 236 246 236
query11 17176 14335 14229 14229
query12 138 90 84 84
query13 1630 414 399 399
query14 10900 10799 10297 10297
query15 264 211 191 191
query16 7976 248 242 242
query17 1968 541 543 541
query18 1975 270 262 262
query19 215 148 143 143
query20 85 86 83 83
query21 195 127 124 124
query22 4653 4437 4520 4437
query23 31911 30902 30841 30841
query24 10822 2861 2800 2800
query25 614 378 372 372
query26 1214 148 150 148
query27 2795 348 349 348
query28 7918 1868 1881 1868
query29 882 636 616 616
query30 298 147 144 144
query31 974 741 719 719
query32 94 61 54 54
query33 749 342 336 336
query34 915 455 468 455
query35 817 592 596 592
query36 990 866 862 862
query37 114 81 76 76
query38 3501 3430 3412 3412
query39 1548 1358 1489 1358
query40 211 117 108 108
query41 49 46 49 46
query42 99 98 97 97
query43 498 460 471 460
query44 1155 710 714 710
query45 281 264 259 259
query46 1087 679 668 668
query47 1669 1607 1595 1595
query48 435 353 361 353
query49 1137 330 334 330
query50 751 373 356 356
query51 6678 6678 6610 6610
query52 107 95 85 85
query53 345 270 275 270
query54 300 249 258 249
query55 86 86 80 80
query56 266 243 220 220
query57 1077 1015 1003 1003
query58 232 203 213 203
query59 2839 2572 2630 2572
query60 273 243 252 243
query61 115 111 110 110
query62 596 400 406 400
query63 302 274 271 271
query64 5574 3985 3880 3880
query65 3029 2991 2998 2991
query66 871 371 355 355
query67 14799 14291 14272 14272
query68 8777 538 519 519
query69 631 398 371 371
query70 1234 1186 1173 1173
query71 525 283 289 283
query72 6379 2618 2500 2500
query73 741 307 311 307
query74 8286 6571 6625 6571
query75 4631 2848 2900 2848
query76 5301 847 913 847
query77 685 258 248 248
query78 10223 9770 9479 9479
query79 8207 511 509 509
query80 1515 380 373 373
query81 513 210 229 210
query82 847 205 212 205
query83 184 140 137 137
query84 282 78 78 78
query85 1446 314 328 314
query86 415 292 306 292
query87 3746 3538 3537 3537
query88 4982 2287 2275 2275
query89 504 356 361 356
query90 1997 174 176 174
query91 164 130 130 130
query92 63 47 47 47
query93 6495 489 471 471
query94 1107 172 170 170
query95 422 324 321 321
query96 599 269 270 269
query97 3020 2869 2871 2869
query98 226 213 198 198
query99 1092 746 754 746
Total cold run time: 309405 ms
Total hot run time: 180165 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2017733969
<details>
<summary>TPC-H: <b>Total hot run time: 38504 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d2c22a6d12f8b88a2adb4b45f5941f5252da8107, data reload: false
------ Round 1 ----------------------------------
q1 17634 4463 4206 4206
q2 2532 176 173 173
q3 11370 1121 1178 1121
q4 10896 825 858 825
q5 7589 3066 3085 3066
q6 212 127 126 126
q7 1065 620 614 614
q8 9517 2059 2047 2047
q9 7341 6761 6737 6737
q10 8751 3467 3569 3467
q11 438 229 217 217
q12 366 203 195 195
q13 17819 2841 2862 2841
q14 222 201 206 201
q15 514 448 451 448
q16 492 374 373 373
q17 979 587 608 587
q18 7220 6455 6542 6455
q19 3397 1380 1447 1380
q20 562 272 249 249
q21 3588 2871 3040 2871
q22 359 312 305 305
Total cold run time: 112863 ms
Total hot run time: 38504 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4105 4077 4049 4049
q2 327 243 241 241
q3 3066 2914 2850 2850
q4 1830 1606 1532 1532
q5 5372 5451 5342 5342
q6 198 117 117 117
q7 2231 1891 1879 1879
q8 3169 3299 3301 3299
q9 8727 8738 8709 8709
q10 3838 3762 3816 3762
q11 534 440 452 440
q12 730 531 530 530
q13 16926 2896 2865 2865
q14 272 239 267 239
q15 501 457 462 457
q16 489 421 409 409
q17 1737 1505 1475 1475
q18 7405 7155 7207 7155
q19 1735 1527 1537 1527
q20 1895 1719 1720 1719
q21 4793 4689 4944 4689
q22 514 427 447 427
Total cold run time: 70394 ms
Total hot run time: 53712 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2012162860
<details>
<summary>TPC-DS: <b>Total hot run time: 181426 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d255d816b72a7f1d0b275d255d866917191f20dd, data reload: false
query1 919 361 335 335
query2 7361 2045 1996 1996
query3 6709 211 208 208
query4 31324 20657 20632 20632
query5 4550 401 482 401
query6 268 172 173 172
query7 4618 299 285 285
query8 236 177 174 174
query9 9953 2286 2280 2280
query10 461 247 274 247
query11 14711 14253 14108 14108
query12 145 92 86 86
query13 1632 421 421 421
query14 11866 11288 11173 11173
query15 269 206 199 199
query16 8179 264 261 261
query17 1958 586 542 542
query18 2099 286 279 279
query19 332 156 168 156
query20 92 90 92 90
query21 202 129 127 127
query22 4653 4448 4413 4413
query23 31804 31028 31247 31028
query24 11519 2819 2783 2783
query25 652 398 386 386
query26 1762 157 159 157
query27 2961 344 345 344
query28 8037 1927 1898 1898
query29 1015 673 621 621
query30 307 148 147 147
query31 963 724 721 721
query32 98 59 56 56
query33 772 268 266 266
query34 1035 477 473 473
query35 830 607 613 607
query36 1005 880 868 868
query37 156 78 77 77
query38 3583 3423 3431 3423
query39 1469 1364 1370 1364
query40 296 119 117 117
query41 53 47 49 47
query42 109 95 103 95
query43 484 448 447 447
query44 1138 722 703 703
query45 260 261 263 261
query46 1099 680 678 678
query47 1676 1600 1602 1600
query48 438 365 358 358
query49 1224 355 337 337
query50 757 369 379 369
query51 6702 6645 6604 6604
query52 114 90 100 90
query53 342 282 274 274
query54 320 256 250 250
query55 85 79 82 79
query56 266 313 237 237
query57 1115 1017 1018 1017
query58 229 208 213 208
query59 2714 2501 2625 2501
query60 283 252 268 252
query61 120 120 118 118
query62 615 401 387 387
query63 300 282 287 282
query64 6814 3827 3629 3629
query65 3138 3027 3031 3027
query66 1402 346 336 336
query67 15065 14348 14286 14286
query68 7771 520 525 520
query69 612 373 396 373
query70 1270 1181 1132 1132
query71 523 277 280 277
query72 6529 2670 2544 2544
query73 749 313 320 313
query74 7649 6585 6608 6585
query75 4311 2892 2837 2837
query76 4977 912 911 911
query77 647 271 250 250
query78 10456 9643 9525 9525
query79 10123 512 525 512
query80 1721 388 389 388
query81 523 207 209 207
query82 883 203 204 203
query83 216 143 148 143
query84 284 78 81 78
query85 1517 325 311 311
query86 476 305 316 305
query87 3738 3540 3534 3534
query88 5091 2307 2302 2302
query89 503 364 366 364
query90 1994 175 176 175
query91 161 129 131 129
query92 59 46 47 46
query93 7103 509 481 481
query94 1239 178 173 173
query95 424 327 325 325
query96 616 270 274 270
query97 3067 2871 2872 2871
query98 250 205 203 203
query99 1241 769 746 746
Total cold run time: 314386 ms
Total hot run time: 181426 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2012170965
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit d255d816b72a7f1d0b275d255d866917191f20dd with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 22.5 seconds inserted 10000000 Rows, about 444K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2012135493
<details>
<summary>TPC-H: <b>Total hot run time: 38183 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d255d816b72a7f1d0b275d255d866917191f20dd, data reload: false
------ Round 1 ----------------------------------
q1 17616 4251 4130 4130
q2 2023 161 149 149
q3 10942 1209 1236 1209
q4 10575 794 767 767
q5 7544 3056 3036 3036
q6 205 126 126 126
q7 1069 652 584 584
q8 10278 2060 2025 2025
q9 7219 6599 6555 6555
q10 8991 3466 3592 3466
q11 548 226 220 220
q12 644 211 205 205
q13 19036 2872 2874 2872
q14 225 199 210 199
q15 502 452 456 452
q16 456 354 354 354
q17 959 515 645 515
q18 7224 6515 6502 6502
q19 2807 1437 1462 1437
q20 515 258 246 246
q21 3589 2975 2839 2839
q22 352 295 296 295
Total cold run time: 113319 ms
Total hot run time: 38183 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4172 4080 4095 4080
q2 333 241 227 227
q3 3014 2866 2844 2844
q4 1814 1587 1540 1540
q5 5216 5264 5281 5264
q6 199 117 119 117
q7 2226 1862 1873 1862
q8 3167 3272 3273 3272
q9 8564 8545 8525 8525
q10 3704 3715 3684 3684
q11 535 434 434 434
q12 718 603 558 558
q13 16889 2859 2841 2841
q14 272 255 273 255
q15 469 447 456 447
q16 459 418 407 407
q17 1733 1476 1465 1465
q18 7419 7144 7138 7138
q19 1612 1562 1527 1527
q20 1887 1698 1705 1698
q21 4738 4655 4771 4655
q22 510 448 466 448
Total cold run time: 69650 ms
Total hot run time: 53288 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "SWJTU-ZhangLei (via GitHub)" <gi...@apache.org>.
SWJTU-ZhangLei commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2017703487
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "SWJTU-ZhangLei (via GitHub)" <gi...@apache.org>.
SWJTU-ZhangLei commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2017126001
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "SWJTU-ZhangLei (via GitHub)" <gi...@apache.org>.
SWJTU-ZhangLei commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2009435042
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "SWJTU-ZhangLei (via GitHub)" <gi...@apache.org>.
SWJTU-ZhangLei commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2011056902
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "SWJTU-ZhangLei (via GitHub)" <gi...@apache.org>.
SWJTU-ZhangLei commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2011662782
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2017760484
<details>
<summary>TPC-DS: <b>Total hot run time: 186110 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d2c22a6d12f8b88a2adb4b45f5941f5252da8107, data reload: false
query1 944 375 359 359
query2 7541 1915 1939 1915
query3 6712 212 209 209
query4 31489 21601 21277 21277
query5 4343 403 403 403
query6 265 182 184 182
query7 4627 295 293 293
query8 224 188 177 177
query9 9150 2293 2253 2253
query10 569 251 252 251
query11 14665 14318 14188 14188
query12 144 93 90 90
query13 1627 439 425 425
query14 12300 11504 11348 11348
query15 252 216 196 196
query16 8022 273 257 257
query17 1949 592 556 556
query18 2053 295 287 287
query19 252 162 159 159
query20 94 86 91 86
query21 199 129 134 129
query22 5005 4817 4786 4786
query23 33390 32480 32543 32480
query24 11683 2828 2836 2828
query25 655 388 381 381
query26 1793 153 150 150
query27 3017 351 350 350
query28 7600 1868 1825 1825
query29 1037 645 609 609
query30 303 146 145 145
query31 949 719 730 719
query32 92 55 56 55
query33 770 241 247 241
query34 1068 485 486 485
query35 832 613 614 613
query36 1016 884 893 884
query37 173 74 75 74
query38 3591 3467 3398 3398
query39 1456 1436 1404 1404
query40 291 110 111 110
query41 49 46 46 46
query42 102 94 95 94
query43 474 441 435 435
query44 1227 732 718 718
query45 275 252 255 252
query46 1104 689 699 689
query47 1914 1863 1848 1848
query48 459 351 347 347
query49 1214 330 339 330
query50 750 371 372 371
query51 6686 6714 6591 6591
query52 116 84 98 84
query53 345 276 275 275
query54 311 234 242 234
query55 85 78 80 78
query56 235 226 220 220
query57 1201 1131 1141 1131
query58 227 204 207 204
query59 2692 2617 2616 2616
query60 260 242 241 241
query61 99 102 120 102
query62 659 445 445 445
query63 314 283 275 275
query64 6887 3964 4092 3964
query65 3132 3064 3072 3064
query66 1427 383 374 374
query67 15405 14796 15020 14796
query68 5322 511 521 511
query69 566 385 387 385
query70 1181 1159 1130 1130
query71 420 274 273 273
query72 6891 2852 2675 2675
query73 706 316 318 316
query74 7773 6363 6447 6363
query75 3517 2849 2839 2839
query76 3821 877 890 877
query77 599 274 257 257
query78 10915 10225 10220 10220
query79 8603 528 521 521
query80 1889 374 373 373
query81 544 210 217 210
query82 1173 213 210 210
query83 321 144 177 144
query84 289 81 78 78
query85 1674 322 318 318
query86 487 291 277 277
query87 3769 3550 3565 3550
query88 5029 2278 2269 2269
query89 542 373 375 373
query90 1973 175 172 172
query91 167 134 140 134
query92 59 47 46 46
query93 6341 498 471 471
query94 1234 173 172 172
query95 427 331 328 328
query96 595 271 266 266
query97 3089 2869 2867 2867
query98 223 218 205 205
query99 1180 879 899 879
Total cold run time: 311110 ms
Total hot run time: 186110 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2018227560
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "SWJTU-ZhangLei (via GitHub)" <gi...@apache.org>.
SWJTU-ZhangLei commented on code in PR #32556:
URL: https://github.com/apache/doris/pull/32556#discussion_r1537557049
##########
fe/fe-core/src/main/java/org/apache/doris/metric/MetricCalculator.java:
##########
@@ -77,4 +87,69 @@ private void update() {
}
MetricRepo.GAUGE_MAX_TABLET_COMPACTION_SCORE.setValue(maxCompactionScore);
}
+
+ private void initCloudMetrics() {
+ if (!Config.isCloudMode()) {
Review Comment:
> ditto
resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2011728105
<details>
<summary>TPC-H: <b>Total hot run time: 37360 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d875478d466f8b421e8d9209c18dc5187f337f89, data reload: false
------ Round 1 ----------------------------------
q1 17666 4220 4055 4055
q2 2018 151 156 151
q3 10587 1119 1183 1119
q4 10231 784 808 784
q5 7462 2948 2912 2912
q6 202 123 124 123
q7 1043 574 569 569
q8 9362 1978 1970 1970
q9 7087 6491 6399 6399
q10 8403 3313 3418 3313
q11 433 219 220 219
q12 408 208 201 201
q13 17791 2852 2861 2852
q14 226 220 215 215
q15 508 457 458 457
q16 455 358 355 355
q17 946 582 606 582
q18 7144 6372 6349 6349
q19 5215 1380 1442 1380
q20 538 263 246 246
q21 3534 2879 2814 2814
q22 366 295 318 295
Total cold run time: 111625 ms
Total hot run time: 37360 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4144 4078 4083 4078
q2 323 226 242 226
q3 2946 2850 2880 2850
q4 1843 1501 1532 1501
q5 5220 5233 5231 5231
q6 194 119 119 119
q7 2220 1883 1829 1829
q8 3138 3267 3260 3260
q9 8590 8513 8520 8513
q10 3709 3652 3615 3615
q11 535 427 443 427
q12 744 513 553 513
q13 16938 2860 2858 2858
q14 279 247 272 247
q15 497 438 448 438
q16 456 413 418 413
q17 1727 1467 1443 1443
q18 7403 7055 7050 7050
q19 2243 1508 1504 1504
q20 1903 1686 1681 1681
q21 4788 4781 4783 4781
q22 527 439 442 439
Total cold run time: 70367 ms
Total hot run time: 53016 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "SWJTU-ZhangLei (via GitHub)" <gi...@apache.org>.
SWJTU-ZhangLei commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2009202122
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2009201353
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2017146976
<details>
<summary>TPC-DS: <b>Total hot run time: 185981 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit db8e17a26643208d87ef84b2221608f6baf24b34, data reload: false
query1 932 371 350 350
query2 7377 2019 1992 1992
query3 6704 216 222 216
query4 31773 21285 21371 21285
query5 4324 462 424 424
query6 291 171 172 171
query7 4621 292 286 286
query8 240 163 172 163
query9 9201 2251 2250 2250
query10 559 245 248 245
query11 17494 14408 14484 14408
query12 133 90 84 84
query13 1625 409 414 409
query14 11160 11445 10669 10669
query15 287 207 196 196
query16 8151 259 257 257
query17 1951 595 564 564
query18 2080 294 290 290
query19 300 160 160 160
query20 102 89 92 89
query21 217 130 129 129
query22 5014 4750 4741 4741
query23 33691 32585 32928 32585
query24 10713 2830 2833 2830
query25 600 386 397 386
query26 781 153 159 153
query27 2432 354 352 352
query28 6685 1894 1843 1843
query29 917 667 592 592
query30 300 149 150 149
query31 985 731 724 724
query32 94 58 56 56
query33 772 257 252 252
query34 1103 489 495 489
query35 814 604 608 604
query36 1016 866 855 855
query37 118 75 76 75
query38 3601 3399 3452 3399
query39 1489 1422 1417 1417
query40 218 110 109 109
query41 49 46 44 44
query42 102 93 95 93
query43 466 425 451 425
query44 1181 723 714 714
query45 275 242 264 242
query46 1118 702 703 702
query47 1918 1831 1860 1831
query48 432 351 351 351
query49 1099 334 330 330
query50 751 376 369 369
query51 6738 6634 6627 6627
query52 105 86 98 86
query53 340 276 274 274
query54 307 249 241 241
query55 83 78 80 78
query56 254 224 223 223
query57 1211 1132 1137 1132
query58 237 198 206 198
query59 2745 2627 2546 2546
query60 256 237 248 237
query61 96 96 96 96
query62 668 442 463 442
query63 308 282 274 274
query64 5579 4020 4121 4020
query65 3087 3009 3039 3009
query66 883 383 382 382
query67 15292 14924 14731 14731
query68 5254 522 519 519
query69 566 380 385 380
query70 1254 1189 1181 1181
query71 425 282 287 282
query72 6450 2876 2703 2703
query73 697 312 326 312
query74 7916 6767 6655 6655
query75 3557 2879 2798 2798
query76 3701 878 869 869
query77 625 275 278 275
query78 10826 10161 10240 10161
query79 8739 513 517 513
query80 2010 381 374 374
query81 555 212 220 212
query82 1417 203 193 193
query83 315 142 141 141
query84 287 79 80 79
query85 1720 332 309 309
query86 492 297 301 297
query87 3777 3542 3542 3542
query88 5081 2282 2264 2264
query89 559 373 363 363
query90 1957 185 177 177
query91 168 138 137 137
query92 59 47 47 47
query93 6660 484 480 480
query94 1232 177 172 172
query95 426 336 325 325
query96 605 268 275 268
query97 3075 2879 2925 2879
query98 228 222 216 216
query99 1232 921 913 913
Total cold run time: 308310 ms
Total hot run time: 185981 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2017138419
<details>
<summary>TPC-H: <b>Total hot run time: 38459 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit db8e17a26643208d87ef84b2221608f6baf24b34, data reload: false
------ Round 1 ----------------------------------
q1 18026 6022 4298 4298
q2 2473 166 155 155
q3 11700 1138 1265 1138
q4 10716 753 755 753
q5 7846 3166 3117 3117
q6 210 127 123 123
q7 1053 610 578 578
q8 9641 1999 2041 1999
q9 7403 6765 6779 6765
q10 9144 3450 3543 3450
q11 425 223 231 223
q12 379 200 199 199
q13 17807 2898 2853 2853
q14 245 201 199 199
q15 501 458 462 458
q16 498 385 373 373
q17 959 534 587 534
q18 7223 6515 6433 6433
q19 1858 1399 1427 1399
q20 550 252 249 249
q21 3566 2867 3018 2867
q22 355 296 297 296
Total cold run time: 112578 ms
Total hot run time: 38459 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4179 4172 4094 4094
q2 327 237 229 229
q3 2990 2879 2864 2864
q4 1859 1571 1614 1571
q5 5288 5361 5341 5341
q6 194 118 117 117
q7 2241 1859 1817 1817
q8 3173 3329 3325 3325
q9 8726 8672 8765 8672
q10 3785 3774 3765 3765
q11 552 441 443 441
q12 710 569 549 549
q13 16898 2870 2881 2870
q14 284 252 259 252
q15 500 466 462 462
q16 473 417 433 417
q17 1759 1520 1490 1490
q18 7596 7182 7232 7182
q19 1633 1571 1569 1569
q20 1901 1728 1735 1728
q21 4938 4835 4712 4712
q22 540 454 474 454
Total cold run time: 70546 ms
Total hot run time: 53921 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #32556:
URL: https://github.com/apache/doris/pull/32556#discussion_r1537240582
##########
fe/fe-core/src/main/java/org/apache/doris/metric/CloudMetrics.java:
##########
@@ -0,0 +1,110 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.metric;
+
+import org.apache.doris.common.Config;
+import org.apache.doris.metric.Metric.MetricUnit;
+
+import com.codahale.metrics.Histogram;
+import com.codahale.metrics.MetricRegistry;
+
+public class CloudMetrics {
+ protected static String CLOUD_CLUSTER_DELIMITER = "@delimiter#";
+ protected static AutoMappedMetric<LongCounterMetric> CLUSTER_REQUEST_ALL_COUNTER;
+ protected static AutoMappedMetric<LongCounterMetric> CLUSTER_QUERY_ALL_COUNTER;
+ protected static AutoMappedMetric<LongCounterMetric> CLUSTER_QUERY_ERR_COUNTER;
+
+ protected static AutoMappedMetric<GaugeMetricImpl<Double>> CLUSTER_REQUEST_PER_SECOND_GAUGE;
+ protected static AutoMappedMetric<GaugeMetricImpl<Double>> CLUSTER_QUERY_PER_SECOND_GAUGE;
+ protected static AutoMappedMetric<GaugeMetricImpl<Double>> CLUSTER_QUERY_ERR_RATE_GAUGE;
+
+ protected static AutoMappedMetric<Histogram> CLUSTER_QUERY_LATENCY_HISTO;
+
+ protected static AutoMappedMetric<GaugeMetricImpl<Integer>> CLUSTER_BACKEND_ALIVE;
+ protected static AutoMappedMetric<GaugeMetricImpl<Integer>> CLUSTER_BACKEND_ALIVE_TOTAL;
+
+ protected static void init() {
+ if (!Config.isCloudMode()) {
Review Comment:
Config.isNotCloudMode()
##########
fe/fe-core/src/main/java/org/apache/doris/metric/MetricCalculator.java:
##########
@@ -77,4 +87,69 @@ private void update() {
}
MetricRepo.GAUGE_MAX_TABLET_COMPACTION_SCORE.setValue(maxCompactionScore);
}
+
+ private void initCloudMetrics() {
+ if (!Config.isCloudMode()) {
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-cloud) Add several multiple dimensions cloud metrics [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32556:
URL: https://github.com/apache/doris/pull/32556#issuecomment-2020543678
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org