You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "XieJiann (via GitHub)" <gi...@apache.org> on 2024/01/11 07:41:48 UTC
[PR] [feat](Nereids): extract agg in struct info node [doris]
XieJiann opened a new pull request, #29853:
URL: https://github.com/apache/doris/pull/29853
## Proposed changes
extract agg in struct info node
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](Nereids): extract agg in struct info node [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29853:
URL: https://github.com/apache/doris/pull/29853#issuecomment-1886584280
<details>
<summary>TPC-H: <b>Total hot run time: 38914 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1930ca4cb7d7fc47150a0895c9924c00d524a5d0, data reload: false
------ Round 1 ----------------------------------
q1 18790 5145 5086 5086
q2 2144 146 138 138
q3 10683 1123 1158 1123
q4 10246 815 801 801
q5 7781 3238 3190 3190
q6 198 123 125 123
q7 853 503 486 486
q8 9277 1961 1974 1961
q9 7354 6517 6445 6445
q10 8254 3115 3050 3050
q11 428 212 200 200
q12 357 189 190 189
q13 18108 3424 3437 3424
q14 247 210 214 210
q15 559 518 505 505
q16 430 384 383 383
q17 946 531 531 531
q18 7289 6962 6655 6655
q19 1584 1386 1422 1386
q20 551 319 311 311
q21 2779 2466 2422 2422
q22 354 298 295 295
Total cold run time: 109212 ms
Total hot run time: 38914 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4946 5011 4980 4980
q2 314 197 204 197
q3 3335 3299 3315 3299
q4 2261 2243 2264 2243
q5 5805 5804 5811 5804
q6 192 117 118 117
q7 2325 1888 1850 1850
q8 3454 3550 3563 3550
q9 8863 8856 8853 8853
q10 3735 3836 3821 3821
q11 557 439 434 434
q12 782 600 614 600
q13 3905 3212 3206 3206
q14 287 264 259 259
q15 573 517 505 505
q16 507 468 466 466
q17 2058 2041 2035 2035
q18 8735 8335 8326 8326
q19 1606 1576 1629 1576
q20 2168 1950 1914 1914
q21 6103 5681 5673 5673
q22 543 462 443 443
Total cold run time: 63054 ms
Total hot run time: 60151 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](Nereids): extract agg in struct info node [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29853:
URL: https://github.com/apache/doris/pull/29853#issuecomment-1891264164
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](Nereids): extract agg in struct info node [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29853:
URL: https://github.com/apache/doris/pull/29853#issuecomment-1891328230
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](Nereids): extract agg in struct info node [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #29853:
URL: https://github.com/apache/doris/pull/29853#issuecomment-1886510313
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](Nereids): extract agg in struct info node [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29853:
URL: https://github.com/apache/doris/pull/29853#issuecomment-1886608738
<details>
<summary>ClickBench: <b>Total hot run time: 30.3 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1930ca4cb7d7fc47150a0895c9924c00d524a5d0, data reload: false
query1 0.06 0.06 0.05
query2 0.06 0.02 0.02
query3 0.26 0.11 0.12
query4 1.73 0.12 0.12
query5 0.52 0.52 0.52
query6 1.40 0.65 0.64
query7 0.01 0.01 0.01
query8 0.03 0.02 0.02
query9 0.57 0.51 0.50
query10 0.57 0.54 0.55
query11 0.13 0.09 0.09
query12 0.12 0.10 0.09
query13 0.60 0.60 0.60
query14 0.76 0.80 0.79
query15 0.82 0.80 0.82
query16 0.35 0.36 0.35
query17 0.94 0.97 0.99
query18 0.26 0.25 0.25
query19 1.84 1.78 1.77
query20 0.01 0.01 0.01
query21 15.42 0.58 0.59
query22 2.45 2.42 1.81
query23 17.08 0.94 0.63
query24 16.22 0.60 0.60
query25 2.17 0.15 0.15
query26 0.13 0.14 0.13
query27 0.14 0.15 0.14
query28 7.20 0.85 0.79
query29 12.64 3.17 3.11
query30 0.54 0.46 0.49
query31 2.78 0.36 0.36
query32 3.36 0.48 0.49
query33 3.23 3.24 3.24
query34 15.87 4.12 4.13
query35 4.14 4.16 4.14
query36 1.10 1.06 1.05
query37 0.07 0.05 0.05
query38 0.04 0.03 0.02
query39 0.03 0.02 0.02
query40 0.16 0.14 0.13
query41 0.08 0.01 0.02
query42 0.03 0.01 0.02
query43 0.03 0.02 0.02
Total cold run time: 115.95 s
Total hot run time: 30.3 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](Nereids): extract agg in struct info node [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #29853:
URL: https://github.com/apache/doris/pull/29853#issuecomment-1886541470
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](Nereids): extract agg in struct info node [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29853:
URL: https://github.com/apache/doris/pull/29853#issuecomment-1886612903
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 1930ca4cb7d7fc47150a0895c9924c00d524a5d0 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 12.7 seconds inserted 10000000 Rows, about 787K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](Nereids): extract agg in struct info node [doris]
Posted by "jackwener (via GitHub)" <gi...@apache.org>.
jackwener merged PR #29853:
URL: https://github.com/apache/doris/pull/29853
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](Nereids): extract agg in struct info node [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29853:
URL: https://github.com/apache/doris/pull/29853#issuecomment-1886600816
<details>
<summary>TPC-DS: <b>Total hot run time: 177255 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1930ca4cb7d7fc47150a0895c9924c00d524a5d0, data reload: false
query1 939 338 339 338
query2 6705 1810 1862 1810
query3 6711 213 202 202
query4 25907 22323 22142 22142
query5 5706 511 517 511
query6 277 191 185 185
query7 4626 280 272 272
query8 224 205 201 201
query9 9214 2695 2666 2666
query10 656 250 237 237
query11 16195 15400 15265 15265
query12 128 83 76 76
query13 1627 344 348 344
query14 11806 7261 7226 7226
query15 245 193 183 183
query16 6397 281 269 269
query17 1881 507 481 481
query18 1944 296 264 264
query19 287 144 144 144
query20 86 85 77 77
query21 186 91 90 90
query22 4966 4863 4849 4849
query23 31811 30966 30762 30762
query24 12505 2803 2797 2797
query25 566 344 334 334
query26 1715 159 158 158
query27 2881 273 276 273
query28 7180 1889 1885 1885
query29 2034 386 375 375
query30 292 146 144 144
query31 962 780 780 780
query32 94 55 65 55
query33 759 270 267 267
query34 1028 448 432 432
query35 927 746 734 734
query36 1363 1225 1274 1225
query37 184 68 69 68
query38 3420 3300 3263 3263
query39 1317 1277 1283 1277
query40 322 100 91 91
query41 41 35 36 35
query42 100 95 94 94
query43 516 521 487 487
query44 1059 710 731 710
query45 213 185 185 185
query46 1065 654 672 654
query47 1718 1557 1521 1521
query48 395 306 306 306
query49 1288 315 324 315
query50 744 347 358 347
query51 5307 5242 5190 5190
query52 100 100 88 88
query53 219 157 160 157
query54 1453 570 575 570
query55 100 91 83 83
query56 209 201 198 198
query57 965 941 974 941
query58 243 235 217 217
query59 2721 2407 2518 2407
query60 247 248 240 240
query61 92 84 85 84
query62 595 387 424 387
query63 178 148 151 148
query64 5838 1745 1752 1745
query65 3350 3253 3261 3253
query66 1350 332 326 326
query67 15704 15272 15052 15052
query68 12643 537 508 508
query69 587 281 281 281
query70 1708 1593 1528 1528
query71 501 218 221 218
query72 5158 2826 2858 2826
query73 3061 323 312 312
query74 6939 6444 6452 6444
query75 5392 2348 2258 2258
query76 6346 863 1142 863
query77 668 294 258 258
query78 10033 8582 8497 8497
query79 1026 497 513 497
query80 568 352 349 349
query81 455 214 219 214
query82 211 90 88 88
query83 192 146 141 141
query84 266 57 60 57
query85 996 306 287 287
query86 378 389 384 384
query87 3590 3412 3382 3382
query88 2905 2247 2228 2228
query89 344 261 258 258
query90 1805 212 210 210
query91 164 134 134 134
query92 71 53 56 53
query93 1316 493 443 443
query94 841 190 194 190
query95 512 430 436 430
query96 631 315 320 315
query97 4288 4190 4223 4190
query98 195 198 184 184
query99 1017 760 736 736
Total cold run time: 296690 ms
Total hot run time: 177255 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](Nereids): extract agg in struct info node [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29853:
URL: https://github.com/apache/doris/pull/29853#issuecomment-1886602275
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.44 seconds
stream load tsv: 557 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.3 seconds inserted 10000000 Rows, about 469K ops/s
storage size: 17184291854 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org