You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "wuwenchi (via GitHub)" <gi...@apache.org> on 2024/01/28 14:26:56 UTC
[PR] [bugfix](iceberg)fix read NULL with date partition [doris]
wuwenchi opened a new pull request, #30478:
URL: https://github.com/apache/doris/pull/30478
## Proposed changes
1. for `date` type: convert int data to string
2. add test for #30162
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)fix read NULL with date partition [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30478:
URL: https://github.com/apache/doris/pull/30478#issuecomment-1914939887
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)fix read NULL with date partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30478:
URL: https://github.com/apache/doris/pull/30478#issuecomment-1913627665
<details>
<summary>TPC-DS: <b>Total hot run time: 174191 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 197ee0e8fac02d48ef2bda25b70c28224b63a3b0, data reload: false
query1 948 324 328 324
query2 6578 1913 1959 1913
query3 6709 196 192 192
query4 32895 22175 22046 22046
query5 4452 358 349 349
query6 239 154 155 154
query7 4614 266 261 261
query8 235 173 179 173
query9 8531 2283 2270 2270
query10 411 209 213 209
query11 18138 15596 15338 15338
query12 124 69 66 66
query13 1690 367 369 367
query14 9219 6610 6697 6610
query15 218 187 184 184
query16 5695 254 250 250
query17 950 468 454 454
query18 1804 253 247 247
query19 170 130 137 130
query20 74 74 68 68
query21 202 135 129 129
query22 5113 4844 4949 4844
query23 31542 31107 31141 31107
query24 14533 2813 2759 2759
query25 584 309 313 309
query26 1824 147 142 142
query27 3162 288 290 288
query28 8456 1854 1848 1848
query29 2626 627 619 619
query30 1401 144 150 144
query31 2596 717 721 717
query32 74 50 47 47
query33 697 208 212 208
query34 1120 460 459 459
query35 888 793 760 760
query36 1451 1201 1230 1201
query37 91 57 58 57
query38 3264 3174 3146 3146
query39 1326 1242 1250 1242
query40 198 89 81 81
query41 36 35 34 34
query42 81 79 82 79
query43 541 480 491 480
query44 1063 686 698 686
query45 195 177 180 177
query46 1054 660 655 655
query47 1639 1427 1457 1427
query48 394 297 308 297
query49 1140 291 277 277
query50 681 311 313 311
query51 5373 5235 5139 5139
query52 86 84 74 74
query53 321 255 265 255
query54 255 180 180 180
query55 82 74 76 74
query56 171 161 165 161
query57 1017 947 926 926
query58 179 149 158 149
query59 2622 2628 2572 2572
query60 209 181 178 178
query61 83 81 78 78
query62 682 363 342 342
query63 275 251 261 251
query64 4831 1797 1744 1744
query65 3301 3232 3223 3223
query66 1210 333 305 305
query67 14579 14094 14304 14094
query68 5179 492 504 492
query69 475 311 296 296
query70 1648 1617 1556 1556
query71 301 212 204 204
query72 4105 2816 2838 2816
query73 699 312 311 311
query74 6877 6298 6303 6298
query75 2797 2280 2292 2280
query76 3428 1033 978 978
query77 375 237 225 225
query78 9226 8710 8510 8510
query79 1992 499 493 493
query80 1299 335 306 306
query81 537 198 200 198
query82 675 81 77 77
query83 219 117 121 117
query84 278 75 71 71
query85 2336 346 325 325
query86 530 417 392 392
query87 3460 3257 3239 3239
query88 3804 2168 2157 2157
query89 416 362 349 349
query90 1953 192 183 183
query91 150 132 116 116
query92 54 43 43 43
query93 2085 448 415 415
query94 1352 159 160 159
query95 505 451 448 448
query96 634 309 318 309
query97 4251 4124 4090 4090
query98 200 197 184 184
query99 1061 674 729 674
Total cold run time: 289843 ms
Total hot run time: 174191 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)fix read NULL with date partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30478:
URL: https://github.com/apache/doris/pull/30478#issuecomment-1913629184
<details>
<summary>ClickBench: <b>Total hot run time: 30.16 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 197ee0e8fac02d48ef2bda25b70c28224b63a3b0, data reload: false
query1 0.04 0.03 0.03
query2 0.06 0.02 0.02
query3 0.23 0.05 0.05
query4 1.69 0.10 0.10
query5 0.52 0.51 0.52
query6 1.18 0.63 0.62
query7 0.02 0.01 0.01
query8 0.04 0.03 0.03
query9 0.54 0.51 0.47
query10 0.54 0.55 0.57
query11 0.11 0.09 0.08
query12 0.11 0.08 0.09
query13 0.60 0.59 0.61
query14 0.78 0.79 0.79
query15 0.78 0.76 0.79
query16 0.39 0.36 0.38
query17 1.02 1.00 1.00
query18 0.21 0.27 0.23
query19 1.86 1.82 1.76
query20 0.01 0.01 0.01
query21 15.43 0.57 0.56
query22 2.38 2.27 1.46
query23 17.33 0.76 0.76
query24 2.93 0.67 0.92
query25 0.42 0.17 0.10
query26 0.57 0.14 0.13
query27 0.06 0.05 0.04
query28 12.17 0.85 0.83
query29 12.54 3.17 3.12
query30 0.66 0.56 0.58
query31 2.80 0.34 0.35
query32 3.38 0.47 0.47
query33 3.21 3.21 3.20
query34 15.87 4.24 4.33
query35 4.29 4.42 4.33
query36 1.09 1.08 1.05
query37 0.06 0.05 0.05
query38 0.04 0.03 0.03
query39 0.02 0.01 0.02
query40 0.16 0.13 0.12
query41 0.07 0.01 0.02
query42 0.03 0.01 0.02
query43 0.03 0.02 0.02
Total cold run time: 106.27 s
Total hot run time: 30.16 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)fix read NULL with date partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30478:
URL: https://github.com/apache/doris/pull/30478#issuecomment-1913624209
<details>
<summary>TPC-H: <b>Total hot run time: 37544 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 197ee0e8fac02d48ef2bda25b70c28224b63a3b0, data reload: false
------ Round 1 ----------------------------------
q1 17659 4976 4901 4901
q2 2054 137 137 137
q3 10596 957 904 904
q4 4642 749 685 685
q5 7648 2773 2833 2773
q6 182 120 120 120
q7 1105 711 697 697
q8 9262 1986 2012 1986
q9 7202 6344 6330 6330
q10 8087 2447 2416 2416
q11 418 207 215 207
q12 710 286 269 269
q13 17998 3315 3279 3279
q14 268 232 242 232
q15 524 497 484 484
q16 456 409 416 409
q17 928 508 509 508
q18 7031 6311 6149 6149
q19 1570 1388 1376 1376
q20 618 338 334 334
q21 6849 3172 3052 3052
q22 806 296 303 296
Total cold run time: 106613 ms
Total hot run time: 37544 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5239 5012 4913 4913
q2 334 232 226 226
q3 3005 2791 2634 2634
q4 1865 1611 1591 1591
q5 5364 5168 5126 5126
q6 200 115 115 115
q7 2173 1799 1813 1799
q8 3085 3169 3258 3169
q9 8472 8565 10880 8565
q10 5469 3490 3427 3427
q11 539 468 458 458
q12 764 577 609 577
q13 16958 3078 3051 3051
q14 287 250 255 250
q15 532 492 492 492
q16 495 484 484 484
q17 2021 1636 1757 1636
q18 8718 11615 8677 8677
q19 23870 1581 1521 1521
q20 4605 1906 1900 1900
q21 14725 4957 4705 4705
q22 1023 487 456 456
Total cold run time: 109743 ms
Total hot run time: 55772 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)fix read NULL with date partition [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30478:
URL: https://github.com/apache/doris/pull/30478#issuecomment-1914939805
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)fix read NULL with date partition [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #30478:
URL: https://github.com/apache/doris/pull/30478
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)fix read NULL with date partition [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30478:
URL: https://github.com/apache/doris/pull/30478#issuecomment-1913630183
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 197ee0e8fac02d48ef2bda25b70c28224b63a3b0 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 13.6 seconds inserted 10000000 Rows, about 735K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [bugfix](iceberg)fix read NULL with date partition [doris]
Posted by "wuwenchi (via GitHub)" <gi...@apache.org>.
wuwenchi commented on PR #30478:
URL: https://github.com/apache/doris/pull/30478#issuecomment-1913615099
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org