You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "wuwenchi (via GitHub)" <gi...@apache.org> on 2024/03/27 12:58:10 UTC
[PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
wuwenchi opened a new pull request, #32923:
URL: https://github.com/apache/doris/pull/32923
## Proposed changes
Convert the datetime type in the predicate according to the target column.
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32923:
URL: https://github.com/apache/doris/pull/32923#issuecomment-2022739958
<details>
<summary>TPC-H: <b>Total hot run time: 37713 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 56857ffd8e8c5b714713c48f18a220a2b86ef594, data reload: false
------ Round 1 ----------------------------------
q1 17651 4115 4106 4106
q2 2104 150 149 149
q3 10601 1106 1177 1106
q4 10223 717 738 717
q5 7451 3004 2989 2989
q6 203 126 128 126
q7 1021 593 577 577
q8 9344 2004 1979 1979
q9 7251 6598 6592 6592
q10 8416 3416 3534 3416
q11 432 234 218 218
q12 422 194 193 193
q13 17788 2826 2838 2826
q14 239 215 204 204
q15 512 465 462 462
q16 497 381 376 376
q17 944 518 625 518
q18 7103 6503 6356 6356
q19 4009 1482 1372 1372
q20 528 268 255 255
q21 3553 2956 2869 2869
q22 357 309 307 307
Total cold run time: 110649 ms
Total hot run time: 37713 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4139 4069 4038 4038
q2 325 228 224 224
q3 2939 2811 2859 2811
q4 1790 1563 1526 1526
q5 5284 5308 5319 5308
q6 201 117 119 117
q7 2233 1832 1827 1827
q8 3156 3352 3274 3274
q9 8749 8707 8682 8682
q10 3793 3757 3738 3738
q11 547 431 439 431
q12 703 582 557 557
q13 16935 2844 2841 2841
q14 291 253 250 250
q15 502 473 474 473
q16 479 418 423 418
q17 1719 1509 1477 1477
q18 7409 7043 6946 6946
q19 1603 1527 1583 1527
q20 1912 1716 1685 1685
q21 4717 4746 4665 4665
q22 548 470 460 460
Total cold run time: 69974 ms
Total hot run time: 53275 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32923:
URL: https://github.com/apache/doris/pull/32923#issuecomment-2025522420
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "wuwenchi (via GitHub)" <gi...@apache.org>.
wuwenchi commented on PR #32923:
URL: https://github.com/apache/doris/pull/32923#issuecomment-2022706131
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32923:
URL: https://github.com/apache/doris/pull/32923#issuecomment-2025291090
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit dac72a8256c7740ed4cf790d9eaa0d7370645403 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.6 seconds inserted 10000000 Rows, about 735K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32923:
URL: https://github.com/apache/doris/pull/32923#issuecomment-2025272099
<details>
<summary>TPC-DS: <b>Total hot run time: 182590 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dac72a8256c7740ed4cf790d9eaa0d7370645403, data reload: false
query1 931 348 348 348
query2 6354 2096 2075 2075
query3 6712 203 203 203
query4 31732 21273 21436 21273
query5 4326 412 400 400
query6 281 178 174 174
query7 4618 300 301 300
query8 223 167 167 167
query9 9541 2296 2281 2281
query10 416 253 288 253
query11 15303 14254 14183 14183
query12 128 91 87 87
query13 1620 423 439 423
query14 9990 8046 7918 7918
query15 256 187 184 184
query16 8264 261 261 261
query17 1966 598 554 554
query18 2098 286 292 286
query19 350 165 158 158
query20 93 89 87 87
query21 205 139 135 135
query22 5028 4848 4872 4848
query23 33443 32767 33350 32767
query24 10938 2995 2987 2987
query25 616 424 406 406
query26 1089 163 167 163
query27 2532 364 377 364
query28 7280 2020 2012 2012
query29 901 680 650 650
query30 325 156 153 153
query31 1009 790 748 748
query32 100 58 57 57
query33 803 275 260 260
query34 1195 513 515 513
query35 923 728 724 724
query36 1058 905 918 905
query37 138 70 69 69
query38 3698 3674 3640 3640
query39 1087 1070 1073 1070
query40 176 108 113 108
query41 49 47 47 47
query42 113 100 100 100
query43 483 472 473 472
query44 1151 762 773 762
query45 281 270 248 248
query46 1124 729 711 711
query47 2023 1943 1973 1943
query48 463 368 366 366
query49 958 345 342 342
query50 817 402 399 399
query51 6878 6887 6685 6685
query52 107 93 93 93
query53 354 278 291 278
query54 308 244 243 243
query55 86 85 79 79
query56 263 234 231 231
query57 1256 1187 1223 1187
query58 233 234 219 219
query59 2778 2585 2690 2585
query60 253 234 234 234
query61 96 93 92 92
query62 612 432 422 422
query63 306 277 284 277
query64 5316 4050 4055 4050
query65 3087 3026 3043 3026
query66 787 352 350 350
query67 15750 14835 14854 14835
query68 10098 521 522 521
query69 694 389 391 389
query70 1301 1120 1146 1120
query71 542 269 263 263
query72 7545 2521 2370 2370
query73 1701 313 318 313
query74 8064 6477 6401 6401
query75 5120 2229 2237 2229
query76 6913 961 854 854
query77 619 252 260 252
query78 11005 10256 10021 10021
query79 10932 536 524 524
query80 1797 403 403 403
query81 513 214 215 214
query82 168 90 83 83
query83 227 147 148 147
query84 286 76 72 72
query85 1135 311 302 302
query86 355 291 313 291
query87 3793 3575 3536 3536
query88 4596 2330 2309 2309
query89 500 363 366 363
query90 2041 180 177 177
query91 172 134 136 134
query92 55 45 46 45
query93 5001 498 496 496
query94 1285 175 178 175
query95 418 310 313 310
query96 614 274 266 266
query97 2632 2499 2520 2499
query98 228 216 224 216
query99 1060 844 825 825
Total cold run time: 315088 ms
Total hot run time: 182590 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32923:
URL: https://github.com/apache/doris/pull/32923#issuecomment-2022789544
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 56857ffd8e8c5b714713c48f18a220a2b86ef594 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.8 seconds inserted 10000000 Rows, about 724K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "wuwenchi (via GitHub)" <gi...@apache.org>.
wuwenchi commented on PR #32923:
URL: https://github.com/apache/doris/pull/32923#issuecomment-2025185602
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #32923:
URL: https://github.com/apache/doris/pull/32923
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32923:
URL: https://github.com/apache/doris/pull/32923#issuecomment-2025522348
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32923:
URL: https://github.com/apache/doris/pull/32923#issuecomment-2022705918
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32923:
URL: https://github.com/apache/doris/pull/32923#issuecomment-2022769329
<details>
<summary>TPC-DS: <b>Total hot run time: 181539 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 56857ffd8e8c5b714713c48f18a220a2b86ef594, data reload: false
query1 964 370 354 354
query2 6550 1941 1863 1863
query3 6707 212 215 212
query4 31844 21273 21356 21273
query5 4279 398 395 395
query6 266 185 180 180
query7 4626 295 284 284
query8 239 170 172 170
query9 9424 2340 2306 2306
query10 565 250 279 250
query11 15283 14255 14242 14242
query12 135 94 87 87
query13 1626 421 416 416
query14 10661 8065 7446 7446
query15 259 209 200 200
query16 8034 255 251 251
query17 2120 559 545 545
query18 1389 301 274 274
query19 348 151 154 151
query20 94 82 94 82
query21 204 126 126 126
query22 4941 4848 4769 4769
query23 33414 32842 32715 32715
query24 10973 2918 2839 2839
query25 638 389 394 389
query26 1228 159 161 159
query27 2849 355 359 355
query28 7434 1887 1868 1868
query29 906 660 627 627
query30 304 149 151 149
query31 996 728 756 728
query32 104 56 56 56
query33 771 252 252 252
query34 1030 491 498 491
query35 845 645 618 618
query36 1055 921 906 906
query37 138 66 69 66
query38 3560 3478 3437 3437
query39 1472 1463 1432 1432
query40 215 119 114 114
query41 51 48 47 47
query42 100 99 97 97
query43 497 451 450 450
query44 1156 730 748 730
query45 277 257 272 257
query46 1126 698 716 698
query47 1931 1852 1851 1851
query48 455 368 356 356
query49 1115 343 362 343
query50 765 378 388 378
query51 6699 6623 6758 6623
query52 111 87 94 87
query53 348 276 279 276
query54 297 252 243 243
query55 89 79 82 79
query56 247 234 228 228
query57 1206 1152 1158 1152
query58 233 207 217 207
query59 2820 2558 2637 2558
query60 277 241 255 241
query61 115 115 128 115
query62 643 477 448 448
query63 302 274 273 273
query64 5757 4113 4012 4012
query65 3221 3048 3028 3028
query66 971 383 381 381
query67 15374 14796 15175 14796
query68 8994 518 530 518
query69 630 388 392 388
query70 1324 1144 1196 1144
query71 529 262 270 262
query72 6424 2731 2529 2529
query73 1604 318 311 311
query74 7971 6484 6424 6424
query75 3728 2222 2203 2203
query76 5042 937 945 937
query77 652 262 255 255
query78 11057 10229 10157 10157
query79 8779 537 528 528
query80 1796 372 366 366
query81 536 213 221 213
query82 492 86 89 86
query83 212 149 149 149
query84 291 83 79 79
query85 1241 319 311 311
query86 397 293 272 272
query87 3803 3595 3571 3571
query88 4692 2291 2278 2278
query89 482 360 362 360
query90 1945 175 177 175
query91 194 137 136 136
query92 63 49 48 48
query93 6761 496 478 478
query94 1166 174 173 173
query95 444 345 327 327
query96 618 262 266 262
query97 2613 2500 2465 2465
query98 223 222 206 206
query99 1167 901 897 897
Total cold run time: 310543 ms
Total hot run time: 181539 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)Convert the datetime type in the predicate according to the target column [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32923:
URL: https://github.com/apache/doris/pull/32923#issuecomment-2025247398
<details>
<summary>TPC-H: <b>Total hot run time: 37546 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dac72a8256c7740ed4cf790d9eaa0d7370645403, data reload: false
------ Round 1 ----------------------------------
q1 17613 4273 4152 4152
q2 2099 153 151 151
q3 10597 1164 1171 1164
q4 10235 799 759 759
q5 7443 2904 2926 2904
q6 203 125 124 124
q7 1033 578 570 570
q8 9352 1984 1974 1974
q9 6932 6346 6324 6324
q10 8454 3440 3563 3440
q11 436 225 222 222
q12 433 206 198 198
q13 17784 2866 2855 2855
q14 255 205 208 205
q15 508 458 456 456
q16 484 371 370 370
q17 941 601 520 520
q18 7112 6504 6351 6351
q19 2641 1423 1426 1423
q20 543 251 245 245
q21 3570 2860 2882 2860
q22 335 279 287 279
Total cold run time: 109003 ms
Total hot run time: 37546 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4156 4090 4093 4090
q2 326 235 229 229
q3 2945 2832 2766 2766
q4 1830 1556 1555 1555
q5 5162 5225 5196 5196
q6 195 114 120 114
q7 2182 1820 1832 1820
q8 3156 3276 3291 3276
q9 8428 8397 8420 8397
q10 3823 3748 3966 3748
q11 561 465 454 454
q12 754 583 567 567
q13 17046 3058 3024 3024
q14 288 285 266 266
q15 541 480 477 477
q16 497 444 474 444
q17 1784 1537 1515 1515
q18 8011 7552 7453 7453
q19 1944 1587 1566 1566
q20 1996 1900 1830 1830
q21 4843 4847 4896 4847
q22 544 465 461 461
Total cold run time: 71012 ms
Total hot run time: 54095 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org