You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "nextdreamblue (via GitHub)" <gi...@apache.org> on 2024/01/03 11:05:35 UTC
[PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]
nextdreamblue opened a new pull request, #29479:
URL: https://github.com/apache/doris/pull/29479
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29479:
URL: https://github.com/apache/doris/pull/29479
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875211962
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875242362
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 2edc59f1c174d2814e5addb954c49d7a03397683, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5491 5128 5157 5128
q2 390 156 158 156
q3 1459 1186 1199 1186
q4 1096 870 793 793
q5 3115 3086 3116 3086
q6 225 137 132 132
q7 996 551 518 518
q8 2158 2216 2207 2207
q9 6708 6634 6669 6634
q10 3183 3176 3094 3094
q11 325 218 214 214
q12 384 229 243 229
q13 4397 3600 3603 3600
q14 243 225 212 212
q15 600 556 557 556
q16 452 410 391 391
q17 1049 542 487 487
q18 7109 6706 6769 6706
q19 1659 1491 1605 1491
q20 578 342 355 342
q21 2838 2499 2445 2445
q22 399 339 317 317
Total cold run time: 44854 ms
Total hot run time: 39924 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5199 5119 5094 5094
q2 336 240 261 240
q3 3351 3331 3353 3331
q4 2145 2003 1996 1996
q5 5943 5920 5950 5920
q6 232 130 126 126
q7 2358 1909 1868 1868
q8 3582 3650 3630 3630
q9 9052 9015 8980 8980
q10 3878 3936 3939 3936
q11 567 480 481 480
q12 792 646 651 646
q13 3918 3191 3185 3185
q14 313 279 295 279
q15 615 545 528 528
q16 589 531 528 528
q17 2008 1831 1810 1810
q18 8800 8345 8376 8345
q19 1754 1651 1680 1651
q20 2285 2002 1981 1981
q21 5673 5409 5337 5337
q22 582 522 490 490
Total cold run time: 63972 ms
Total hot run time: 60381 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875216832
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 2edc59f1c174d2814e5addb954c49d7a03397683, data reload: false
------ Round 1 ----------------------------------
q1 18761 5445 5046 5046
q2 2035 157 148 148
q3 10605 1159 1093 1093
q4 10204 755 900 755
q5 7785 2946 2977 2946
q6 209 137 131 131
q7 949 517 534 517
q8 9345 2018 2007 2007
q9 6838 6411 6400 6400
q10 8246 3015 3042 3015
q11 430 222 221 221
q12 389 236 231 231
q13 18000 3614 3601 3601
q14 244 222 213 213
q15 574 551 534 534
q16 459 399 401 399
q17 953 526 579 526
q18 7404 6847 6774 6774
q19 1583 1297 1313 1297
q20 744 334 356 334
q21 2863 2345 2346 2345
q22 368 306 331 306
Total cold run time: 108988 ms
Total hot run time: 38839 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5143 5102 5109 5102
q2 338 236 242 236
q3 3324 3292 3265 3265
q4 2162 2045 2022 2022
q5 5826 5815 5779 5779
q6 211 124 121 121
q7 2311 1945 1903 1903
q8 3377 3417 3456 3417
q9 8843 8755 8773 8755
q10 3798 3821 3872 3821
q11 577 481 495 481
q12 823 635 616 616
q13 10699 3196 3190 3190
q14 280 272 267 267
q15 588 531 535 531
q16 580 480 517 480
q17 1941 1777 1762 1762
q18 8759 8449 8255 8255
q19 1617 1621 1618 1618
q20 2254 1987 1970 1970
q21 5502 5286 5242 5242
q22 587 521 492 492
Total cold run time: 69540 ms
Total hot run time: 59325 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]
Posted by "nextdreamblue (via GitHub)" <gi...@apache.org>.
nextdreamblue commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875192616
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875212000
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875230387
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools
```
TPC-DS sf100 test result on commit 2edc59f1c174d2814e5addb954c49d7a03397683, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 889 345 336 336
query2 5122 1907 1873 1873
query3 5185 205 197 197
query4 24009 22452 22288 22288
query5 3235 508 505 505
query6 254 193 188 188
query7 4091 289 294 289
query8 251 223 216 216
query9 7365 2742 2657 2657
query10 337 237 237 237
query11 16026 15362 15731 15362
query12 129 76 73 73
query13 1507 329 319 319
query14 10512 7411 7340 7340
query15 309 218 225 218
query16 5673 269 271 269
query17 1613 523 530 523
query18 1782 289 263 263
query19 180 155 153 153
query20 83 76 82 76
query21 159 101 101 101
query22 5081 4820 4507 4507
query23 32243 31643 31202 31202
query24 11710 3457 3423 3423
query25 624 374 374 374
query26 1603 158 157 157
query27 2709 288 291 288
query28 6277 1966 1942 1942
query29 1810 422 407 407
query30 305 149 156 149
query31 1118 777 820 777
query32 86 59 54 54
query33 673 292 270 270
query34 841 444 435 435
query35 968 783 809 783
query36 1350 1282 1167 1167
query37 175 74 75 74
query38 3456 3306 3274 3274
query39 1341 1307 1285 1285
query40 286 92 93 92
query41 38 36 35 35
query42 102 89 93 89
query43 519 500 479 479
query44 1035 706 717 706
query45 232 224 223 223
query46 1167 781 748 748
query47 1652 1574 1581 1574
query48 370 277 281 277
query49 1139 331 329 329
query50 1077 877 831 831
query51 5361 5261 5418 5261
query52 91 91 90 90
query53 217 185 179 179
query54 1581 607 599 599
query55 95 86 85 85
query56 209 204 192 192
query57 1049 965 996 965
query58 228 216 207 207
query59 2703 2617 2480 2480
query60 244 238 242 238
query61 84 86 87 86
query62 1457 509 507 507
query63 188 173 165 165
query64 5775 1755 1790 1755
query65 3376 3283 3263 3263
query66 1154 344 324 324
query67 15898 15189 15232 15189
query68 11800 539 561 539
query69 498 256 243 243
query70 1695 1531 1546 1531
query71 485 226 224 224
query72 6468 4094 4067 4067
query73 2778 311 322 311
query74 6857 6489 6505 6489
query75 5263 2257 2312 2257
query76 6091 1190 1127 1127
query77 667 293 310 293
query78 9127 8864 8553 8553
query79 5500 536 535 535
query80 747 391 393 391
query81 483 215 215 215
query82 196 103 103 103
query83 185 142 140 140
query84 244 53 52 52
query85 879 281 283 281
query86 397 412 429 412
query87 3546 3522 3394 3394
query88 3713 2292 2289 2289
query89 358 295 293 293
query90 1798 216 210 210
query91 126 93 94 93
query92 59 60 51 51
query93 3396 442 489 442
query94 761 189 188 188
query95 472 405 394 394
query96 636 321 319 319
query97 4262 4179 4150 4150
query98 215 202 190 190
query99 1579 936 889 889
Total cold run time: 289689 ms
Total hot run time: 181684 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](auditloader) support audit table millisecond and fix stmt truncated by '\r' [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29479:
URL: https://github.com/apache/doris/pull/29479#issuecomment-1875240752
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.11 seconds
stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.2 seconds inserted 10000000 Rows, about 367K ops/s
storage size: 17183276427 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org