You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/01/25 10:14:18 UTC
[PR] [feature](journal) Add a method to write a set of journals in batch [doris]
w41ter opened a new pull request, #30380:
URL: https://github.com/apache/doris/pull/30380
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](journal) Add a method to write a set of journals in batch [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #30380:
URL: https://github.com/apache/doris/pull/30380
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](journal) Add a method to write a set of journals in batch [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #30380:
URL: https://github.com/apache/doris/pull/30380#issuecomment-1909825348
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](journal) Add a method to write a set of journals in batch [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30380:
URL: https://github.com/apache/doris/pull/30380#issuecomment-1909926004
<details>
<summary>ClickBench: <b>Total hot run time: 30.86 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 46825895e2ed5d4c9496aa48cc24c6f460006508, data reload: false
query1 0.03 0.03 0.02
query2 0.06 0.02 0.03
query3 0.22 0.05 0.05
query4 1.69 0.08 0.07
query5 0.53 0.53 0.53
query6 1.28 0.64 0.64
query7 0.02 0.01 0.02
query8 0.03 0.03 0.02
query9 0.54 0.50 0.50
query10 0.55 0.55 0.55
query11 0.12 0.08 0.09
query12 0.12 0.09 0.08
query13 0.59 0.62 0.61
query14 0.79 0.80 0.80
query15 0.79 0.78 0.78
query16 0.38 0.36 0.36
query17 1.03 1.00 1.01
query18 0.25 0.24 0.24
query19 1.88 1.79 1.79
query20 0.01 0.01 0.02
query21 15.39 0.58 0.56
query22 2.56 2.10 1.76
query23 17.43 1.09 0.80
query24 2.73 0.88 0.94
query25 0.37 0.23 0.17
query26 0.46 0.14 0.13
query27 0.06 0.06 0.05
query28 11.69 0.78 0.79
query29 12.49 3.23 3.17
query30 0.52 0.48 0.48
query31 2.78 0.34 0.35
query32 3.36 0.48 0.49
query33 3.23 3.29 3.26
query34 15.80 4.34 4.27
query35 4.34 4.35 4.24
query36 1.11 1.06 1.07
query37 0.06 0.06 0.05
query38 0.04 0.03 0.03
query39 0.03 0.02 0.01
query40 0.16 0.14 0.14
query41 0.08 0.01 0.02
query42 0.02 0.01 0.01
query43 0.02 0.02 0.02
Total cold run time: 105.64 s
Total hot run time: 30.86 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](journal) Add a method to write a set of journals in batch [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30380:
URL: https://github.com/apache/doris/pull/30380#issuecomment-1910207891
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](journal) Add a method to write a set of journals in batch [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30380:
URL: https://github.com/apache/doris/pull/30380#issuecomment-1909899616
<details>
<summary>TPC-H: <b>Total hot run time: 38727 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 46825895e2ed5d4c9496aa48cc24c6f460006508, data reload: false
------ Round 1 ----------------------------------
q1 17658 5250 5149 5149
q2 2037 137 133 133
q3 10639 1124 1176 1124
q4 10227 772 779 772
q5 7721 3124 3136 3124
q6 193 122 125 122
q7 867 492 485 485
q8 9205 1915 1936 1915
q9 7270 6386 6334 6334
q10 8237 3026 3031 3026
q11 425 210 198 198
q12 350 188 190 188
q13 18004 3354 3348 3348
q14 239 207 211 207
q15 542 508 505 505
q16 489 369 378 369
q17 937 544 577 544
q18 7396 6958 6810 6810
q19 1545 1403 1402 1402
q20 590 301 303 301
q21 2742 2410 2377 2377
q22 364 294 305 294
Total cold run time: 107677 ms
Total hot run time: 38727 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5300 5354 5247 5247
q2 327 219 212 212
q3 3362 3195 3236 3195
q4 2097 2064 2066 2064
q5 5853 5903 5955 5903
q6 200 114 118 114
q7 2369 1825 1816 1816
q8 3210 3371 3367 3367
q9 8863 8864 8895 8864
q10 3942 3848 3836 3836
q11 558 489 444 444
q12 837 627 634 627
q13 16921 3186 3138 3138
q14 290 253 273 253
q15 543 514 492 492
q16 498 451 439 439
q17 3668 1836 1861 1836
q18 9597 17763 9729 9729
q19 23171 1555 1541 1541
q20 4608 1936 1933 1933
q21 14565 5381 5520 5381
q22 976 542 509 509
Total cold run time: 111755 ms
Total hot run time: 60940 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](journal) Add a method to write a set of journals in batch [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30380:
URL: https://github.com/apache/doris/pull/30380#issuecomment-1909917773
<details>
<summary>TPC-DS: <b>Total hot run time: 187046 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 46825895e2ed5d4c9496aa48cc24c6f460006508, data reload: false
query1 939 345 334 334
query2 6528 1951 1934 1934
query3 6697 203 202 202
query4 32815 22370 22155 22155
query5 4454 452 379 379
query6 244 162 149 149
query7 4609 271 267 267
query8 230 171 174 171
query9 8320 2640 2626 2626
query10 419 227 216 216
query11 16913 15498 15564 15498
query12 123 72 67 67
query13 1679 373 392 373
query14 10605 7148 7070 7070
query15 214 185 187 185
query16 5792 260 248 248
query17 941 477 475 475
query18 1795 261 248 248
query19 182 131 136 131
query20 67 70 67 67
query21 180 134 130 130
query22 4904 4809 4800 4800
query23 32263 30988 31081 30988
query24 5316 2867 2865 2865
query25 407 306 315 306
query26 791 145 157 145
query27 2250 287 291 287
query28 3766 1860 1832 1832
query29 890 622 626 622
query30 195 140 138 138
query31 904 771 772 771
query32 63 51 50 50
query33 376 216 215 215
query34 811 478 472 472
query35 905 809 804 804
query36 1211 1178 1237 1178
query37 98 58 61 58
query38 3383 3249 3203 3203
query39 1324 1268 1268 1268
query40 188 86 80 80
query41 36 40 34 34
query42 88 78 90 78
query43 502 478 449 449
query44 1066 701 710 701
query45 194 178 178 178
query46 1047 658 668 658
query47 1690 1494 1505 1494
query48 392 307 308 307
query49 623 292 293 292
query50 679 318 310 310
query51 5331 5245 5245 5245
query52 90 74 76 74
query53 330 261 260 260
query54 216 181 182 181
query55 76 82 79 79
query56 171 178 173 173
query57 1021 910 931 910
query58 169 161 161 161
query59 2787 2775 2597 2597
query60 212 183 181 181
query61 80 87 79 79
query62 448 375 353 353
query63 288 259 268 259
query64 2547 1822 1801 1801
query65 3350 3305 3268 3268
query66 1090 314 310 310
query67 15541 14847 15165 14847
query68 12557 525 507 507
query69 612 312 308 308
query70 1670 1521 1516 1516
query71 10449 10204 10216 10204
query72 5063 2805 2825 2805
query73 2629 313 307 307
query74 6965 6337 6337 6337
query75 4912 2352 2300 2300
query76 6511 1027 1029 1027
query77 692 232 236 232
query78 9634 8627 8611 8611
query79 1018 495 511 495
query80 606 317 323 317
query81 476 210 201 201
query82 200 79 78 78
query83 136 118 119 118
query84 275 68 68 68
query85 1080 326 318 318
query86 388 376 386 376
query87 3481 3321 3336 3321
query88 3123 2197 2203 2197
query89 420 371 366 366
query90 2256 192 191 191
query91 155 123 120 120
query92 56 43 46 43
query93 1519 422 444 422
query94 1462 163 156 156
query95 506 467 463 463
query96 646 320 325 320
query97 4229 4146 4174 4146
query98 206 188 184 184
query99 959 706 710 706
Total cold run time: 288746 ms
Total hot run time: 187046 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](journal) Add a method to write a set of journals in batch [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30380:
URL: https://github.com/apache/doris/pull/30380#issuecomment-1910305856
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](journal) Add a method to write a set of journals in batch [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30380:
URL: https://github.com/apache/doris/pull/30380#issuecomment-1909930647
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 46825895e2ed5d4c9496aa48cc24c6f460006508 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 14.8 seconds inserted 10000000 Rows, about 675K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org