You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "seawinde (via GitHub)" <gi...@apache.org> on 2023/12/19 14:56:50 UTC
[PR] [feature](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table [doris]
seawinde opened a new pull request, #28678:
URL: https://github.com/apache/doris/pull/28678
## Proposed changes
- Fix data wrong using mv rewrite
- ignore case when getting mv related partition table
this pr is depend on https://github.com/apache/doris/pull/28596
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28678:
URL: https://github.com/apache/doris/pull/28678#issuecomment-1863668781
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.65 seconds
stream load tsv: 576 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17184123405 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28678:
URL: https://github.com/apache/doris/pull/28678#issuecomment-1863015528
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.65 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17184051365 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28678:
URL: https://github.com/apache/doris/pull/28678#issuecomment-1863659752
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Tpch sf100 test result on commit 5b7599afcf35a5ec5a67a202efccd16436071563, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4695 4430 4419 4419
q2 367 148 158 148
q3 1465 1231 1278 1231
q4 1113 907 887 887
q5 3145 3183 3176 3176
q6 249 131 127 127
q7 993 490 486 486
q8 2184 2207 2175 2175
q9 6696 6625 6683 6625
q10 3223 3280 3288 3280
q11 318 191 195 191
q12 362 216 211 211
q13 4561 3870 3783 3783
q14 244 218 216 216
q15 565 524 521 521
q16 455 399 398 398
q17 1004 574 575 574
q18 7214 7069 7042 7042
q19 1513 1396 1463 1396
q20 539 317 331 317
q21 3048 2623 2679 2623
q22 355 289 295 289
Total cold run time: 44308 ms
Total hot run time: 40115 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4399 4367 4372 4367
q2 271 167 178 167
q3 3539 3536 3514 3514
q4 2387 2368 2369 2368
q5 5748 5767 5750 5750
q6 241 120 121 120
q7 2392 1859 1910 1859
q8 3528 3520 3511 3511
q9 9017 8987 9018 8987
q10 3915 4029 4023 4023
q11 490 382 379 379
q12 777 599 608 599
q13 4299 3586 3575 3575
q14 290 255 243 243
q15 582 519 537 519
q16 537 467 494 467
q17 1870 1858 1821 1821
q18 8627 8190 8313 8190
q19 1753 1763 1751 1751
q20 2262 1951 1942 1942
q21 6595 6185 6194 6185
q22 509 452 436 436
Total cold run time: 64028 ms
Total hot run time: 60773 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde closed pull request #28678: [feature](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table
URL: https://github.com/apache/doris/pull/28678
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #28678:
URL: https://github.com/apache/doris/pull/28678#issuecomment-1863634193
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #28678:
URL: https://github.com/apache/doris/pull/28678#issuecomment-1862923653
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table [doris]
Posted by "seawinde (via GitHub)" <gi...@apache.org>.
seawinde commented on PR #28678:
URL: https://github.com/apache/doris/pull/28678#issuecomment-1863169788
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28678:
URL: https://github.com/apache/doris/pull/28678#issuecomment-1863023475
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Tpch sf100 test result on commit f1a5530863b1bee3690b47bea68b7f14842d9e43, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4674 4365 4411 4365
q2 383 173 159 159
q3 1466 1248 1215 1215
q4 1114 917 918 917
q5 3154 3172 3140 3140
q6 246 127 127 127
q7 989 490 481 481
q8 2216 2213 2193 2193
q9 6681 6704 6692 6692
q10 3222 3272 3276 3272
q11 312 182 187 182
q12 350 209 203 203
q13 4536 3780 3792 3780
q14 240 216 211 211
q15 563 520 518 518
q16 433 385 383 383
q17 1006 608 595 595
q18 7275 6974 6942 6942
q19 1517 1409 1385 1385
q20 505 304 297 297
q21 3081 2637 2624 2624
q22 341 281 286 281
Total cold run time: 44304 ms
Total hot run time: 39962 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4331 4337 4341 4337
q2 266 159 166 159
q3 3527 3529 3527 3527
q4 2381 2377 2379 2377
q5 5848 5733 5730 5730
q6 239 118 124 118
q7 2394 1898 1877 1877
q8 3520 3520 3522 3520
q9 9082 9014 9016 9014
q10 3914 4000 3988 3988
q11 479 368 366 366
q12 755 586 608 586
q13 4292 3558 3552 3552
q14 279 255 258 255
q15 569 526 526 526
q16 496 442 466 442
q17 1890 1868 1875 1868
q18 8605 8325 8286 8286
q19 1764 1763 1760 1760
q20 2249 1944 1923 1923
q21 6528 6196 6164 6164
q22 499 417 442 417
Total cold run time: 63907 ms
Total hot run time: 60792 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org