You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "koarz (via GitHub)" <gi...@apache.org> on 2024/01/25 17:47:09 UTC
[PR] [fix](function)make SUBSTRING_INDEX DEPEND_ON_ARGUMENT [doris]
koarz opened a new pull request, #30392:
URL: https://github.com/apache/doris/pull/30392
## Proposed changes
Issue Number: #27435
<!--Describe your changes.-->
make function SUBSTRING_INDEX DEPEND_ON_ARGUMENT
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function)make SUBSTRING_INDEX DEPEND_ON_ARGUMENT [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1910782362
<details>
<summary>ClickBench: <b>Total hot run time: 30.92 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 78b7553d7c0cf4ad323b8095d530f381f2e21dfa, data reload: false
query1 0.03 0.02 0.02
query2 0.06 0.02 0.02
query3 0.22 0.05 0.04
query4 1.71 0.07 0.08
query5 0.54 0.52 0.52
query6 1.32 0.65 0.64
query7 0.02 0.01 0.01
query8 0.04 0.02 0.02
query9 0.56 0.51 0.49
query10 0.57 0.54 0.56
query11 0.12 0.08 0.09
query12 0.11 0.10 0.09
query13 0.61 0.62 0.60
query14 0.77 0.80 0.81
query15 0.79 0.78 0.78
query16 0.38 0.36 0.37
query17 1.05 1.04 0.99
query18 0.22 0.28 0.25
query19 1.87 1.76 1.78
query20 0.01 0.01 0.02
query21 15.40 0.57 0.58
query22 2.27 2.85 1.65
query23 17.29 0.87 0.85
query24 3.50 1.40 1.15
query25 0.38 0.14 0.11
query26 0.69 0.13 0.12
query27 0.05 0.05 0.05
query28 10.36 0.78 0.78
query29 12.52 3.25 3.25
query30 0.52 0.49 0.49
query31 2.78 0.34 0.34
query32 3.39 0.49 0.49
query33 3.17 3.23 3.24
query34 15.78 4.21 4.27
query35 4.28 4.24 4.28
query36 1.09 1.06 1.08
query37 0.06 0.04 0.04
query38 0.04 0.03 0.03
query39 0.02 0.02 0.01
query40 0.16 0.13 0.13
query41 0.07 0.02 0.01
query42 0.02 0.02 0.01
query43 0.02 0.02 0.01
Total cold run time: 104.86 s
Total hot run time: 30.92 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT [doris]
Posted by "koarz (via GitHub)" <gi...@apache.org>.
koarz commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1911484881
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT [doris]
Posted by "koarz (via GitHub)" <gi...@apache.org>.
koarz commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1911571302
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function)make SUBSTRING_INDEX DEPEND_ON_ARGUMENT [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1910757193
<details>
<summary>TPC-H: <b>Total hot run time: 38613 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 78b7553d7c0cf4ad323b8095d530f381f2e21dfa, data reload: false
------ Round 1 ----------------------------------
q1 17741 5323 5220 5220
q2 2052 139 133 133
q3 10738 1172 1184 1172
q4 10269 767 753 753
q5 7731 3106 3140 3106
q6 194 117 119 117
q7 864 497 482 482
q8 9238 1897 1917 1897
q9 7220 6328 6329 6328
q10 8206 3057 3000 3000
q11 410 207 205 205
q12 357 184 192 184
q13 17983 3353 3369 3353
q14 251 207 220 207
q15 542 509 497 497
q16 436 361 373 361
q17 945 535 513 513
q18 7505 6817 6869 6817
q19 1538 1353 1301 1301
q20 586 293 301 293
q21 2721 2448 2374 2374
q22 364 315 300 300
Total cold run time: 107891 ms
Total hot run time: 38613 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5319 5232 5340 5232
q2 327 214 217 214
q3 3282 3227 3229 3227
q4 2053 2050 2051 2050
q5 5954 5903 5911 5903
q6 192 115 116 115
q7 2319 1837 1913 1837
q8 3219 3349 3391 3349
q9 8948 8865 9152 8865
q10 3898 3838 3805 3805
q11 549 464 445 445
q12 788 611 592 592
q13 16929 3155 3172 3155
q14 295 255 283 255
q15 551 511 509 509
q16 506 469 459 459
q17 1905 1809 1843 1809
q18 9515 16893 9538 9538
q19 27193 1550 1514 1514
q20 4608 1946 1919 1919
q21 14411 5248 5132 5132
q22 976 532 520 520
Total cold run time: 113737 ms
Total hot run time: 60444 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1911526089
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function)make SUBSTRING_INDEX DEPEND_ON_ARGUMENT [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1910714714
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee merged PR #30392:
URL: https://github.com/apache/doris/pull/30392
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1911517312
<details>
<summary>TPC-DS: <b>Total hot run time: 187356 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b7ecc51d500aaa808e669a22b2add2d0281bf8e2, data reload: false
query1 940 339 325 325
query2 6554 2031 1977 1977
query3 6696 207 209 207
query4 31992 22244 22292 22244
query5 4458 444 452 444
query6 251 156 154 154
query7 4607 270 265 265
query8 228 180 184 180
query9 8392 2644 2636 2636
query10 432 233 261 233
query11 17173 15522 15594 15522
query12 126 71 70 70
query13 1686 388 379 379
query14 10660 6964 7054 6964
query15 209 179 185 179
query16 5802 254 252 252
query17 955 474 480 474
query18 1782 265 245 245
query19 173 137 133 133
query20 74 76 70 70
query21 188 132 129 129
query22 5033 4913 4944 4913
query23 31742 30867 30832 30832
query24 12522 2800 2774 2774
query25 569 324 315 315
query26 1770 145 144 144
query27 3220 288 288 288
query28 6917 1861 1848 1848
query29 1763 607 618 607
query30 276 136 134 134
query31 927 762 762 762
query32 89 49 50 49
query33 700 217 211 211
query34 1142 454 474 454
query35 879 774 768 768
query36 1361 1228 1214 1214
query37 91 57 60 57
query38 3389 3325 3196 3196
query39 1309 1295 1260 1260
query40 343 89 81 81
query41 38 36 35 35
query42 91 84 84 84
query43 529 462 448 448
query44 1096 694 705 694
query45 195 180 173 173
query46 1072 647 647 647
query47 1708 1608 1572 1572
query48 385 300 304 300
query49 1176 297 295 295
query50 667 313 306 306
query51 5277 5178 5197 5178
query52 93 71 75 71
query53 321 264 260 260
query54 240 180 181 180
query55 79 79 79 79
query56 186 170 168 168
query57 1013 915 922 915
query58 193 157 164 157
query59 2968 2706 2750 2706
query60 208 185 187 185
query61 92 83 85 83
query62 632 366 373 366
query63 272 270 258 258
query64 6156 1768 1754 1754
query65 3398 3274 3232 3232
query66 1265 316 316 316
query67 15637 15126 15317 15126
query68 11029 520 541 520
query69 596 309 282 282
query70 1608 1543 1509 1509
query71 10419 10198 10202 10198
query72 4835 2857 2845 2845
query73 2135 315 314 314
query74 6974 6411 6468 6411
query75 4576 2348 2269 2269
query76 6283 1041 1014 1014
query77 665 235 233 233
query78 9162 8923 8615 8615
query79 1022 494 517 494
query80 539 324 317 317
query81 450 202 202 202
query82 176 78 78 78
query83 137 129 121 121
query84 267 72 71 71
query85 1064 334 346 334
query86 391 384 367 367
query87 3504 3357 3297 3297
query88 3006 2226 2202 2202
query89 437 359 348 348
query90 1950 189 187 187
query91 154 123 132 123
query92 50 43 43 43
query93 2230 430 442 430
query94 1237 160 157 157
query95 503 459 456 456
query96 616 320 331 320
query97 4275 4131 4130 4130
query98 205 191 188 188
query99 1001 705 704 704
Total cold run time: 303903 ms
Total hot run time: 187356 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1911520882
<details>
<summary>ClickBench: <b>Total hot run time: 31.11 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b7ecc51d500aaa808e669a22b2add2d0281bf8e2, data reload: false
query1 0.04 0.02 0.02
query2 0.06 0.03 0.02
query3 0.22 0.05 0.04
query4 1.71 0.07 0.07
query5 0.54 0.52 0.52
query6 1.30 0.63 0.63
query7 0.02 0.01 0.01
query8 0.04 0.02 0.02
query9 0.54 0.51 0.50
query10 0.56 0.56 0.56
query11 0.12 0.09 0.09
query12 0.11 0.09 0.09
query13 0.61 0.62 0.61
query14 0.81 0.81 0.81
query15 0.78 0.78 0.77
query16 0.41 0.39 0.39
query17 1.01 0.97 1.00
query18 0.23 0.28 0.25
query19 1.87 1.76 1.77
query20 0.02 0.01 0.01
query21 15.41 0.58 0.57
query22 2.29 1.89 2.40
query23 17.40 0.82 0.81
query24 2.89 1.12 1.03
query25 0.42 0.16 0.22
query26 0.57 0.13 0.13
query27 0.05 0.06 0.05
query28 10.92 0.78 0.77
query29 12.55 3.19 3.24
query30 0.52 0.47 0.48
query31 2.78 0.36 0.36
query32 3.36 0.48 0.49
query33 3.25 3.23 3.22
query34 16.23 4.27 4.25
query35 4.31 4.28 4.25
query36 1.10 1.08 1.07
query37 0.07 0.05 0.05
query38 0.05 0.03 0.03
query39 0.02 0.01 0.02
query40 0.16 0.14 0.13
query41 0.08 0.02 0.02
query42 0.03 0.02 0.02
query43 0.02 0.02 0.01
Total cold run time: 105.48 s
Total hot run time: 31.11 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function)make SUBSTRING_INDEX DEPEND_ON_ARGUMENT [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1910774570
<details>
<summary>TPC-DS: <b>Total hot run time: 187267 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 78b7553d7c0cf4ad323b8095d530f381f2e21dfa, data reload: false
query1 949 332 337 332
query2 6552 2148 1903 1903
query3 6693 204 195 195
query4 31867 22257 22261 22257
query5 4451 449 419 419
query6 239 154 150 150
query7 4604 263 269 263
query8 235 174 186 174
query9 8349 2602 2583 2583
query10 423 240 228 228
query11 16800 15472 15521 15472
query12 122 68 68 68
query13 1683 371 383 371
query14 10625 6960 6911 6911
query15 205 179 181 179
query16 5796 272 248 248
query17 938 469 477 469
query18 1780 254 253 253
query19 180 129 129 129
query20 67 72 67 67
query21 189 132 128 128
query22 4942 4859 4748 4748
query23 31877 31270 31182 31182
query24 12626 2910 2909 2909
query25 596 341 320 320
query26 1875 147 148 147
query27 3283 313 296 296
query28 7268 1875 1851 1851
query29 2156 656 636 636
query30 303 147 145 145
query31 951 758 805 758
query32 77 54 53 53
query33 714 220 216 216
query34 1310 480 489 480
query35 962 797 755 755
query36 1265 1205 1227 1205
query37 155 63 58 58
query38 3445 3272 3312 3272
query39 1335 1277 1269 1269
query40 1153 90 100 90
query41 36 39 39 39
query42 89 94 91 91
query43 497 493 456 456
query44 1060 690 697 690
query45 194 182 174 174
query46 1085 685 678 678
query47 1582 1586 1625 1586
query48 380 302 309 302
query49 1223 289 290 289
query50 685 316 317 316
query51 5306 5187 5255 5187
query52 98 71 77 71
query53 318 248 265 248
query54 237 184 177 177
query55 78 69 78 69
query56 169 175 155 155
query57 979 908 933 908
query58 195 154 158 154
query59 2903 2652 2441 2441
query60 196 187 176 176
query61 82 82 84 82
query62 616 369 342 342
query63 290 268 252 252
query64 6084 1778 1776 1776
query65 3300 3247 3250 3247
query66 1424 321 322 321
query67 15749 15481 15234 15234
query68 9676 512 519 512
query69 590 304 296 296
query70 1596 1428 1512 1428
query71 10408 10199 10195 10195
query72 4005 2819 2830 2819
query73 1986 324 311 311
query74 6872 6377 6475 6377
query75 4054 2355 2318 2318
query76 5991 1009 956 956
query77 746 226 228 226
query78 9038 9012 8603 8603
query79 995 491 487 487
query80 590 314 319 314
query81 439 207 201 201
query82 326 80 76 76
query83 131 115 116 115
query84 269 66 65 65
query85 1338 336 324 324
query86 408 366 369 366
query87 3544 3361 3335 3335
query88 2711 2202 2189 2189
query89 426 360 352 352
query90 1928 187 188 187
query91 154 130 121 121
query92 50 41 45 41
query93 905 450 435 435
query94 1017 158 156 156
query95 507 464 448 448
query96 614 317 333 317
query97 4304 4157 4152 4152
query98 204 191 193 191
query99 1238 712 704 704
Total cold run time: 300955 ms
Total hot run time: 187267 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function)make SUBSTRING_INDEX DEPEND_ON_ARGUMENT [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1910719533
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on code in PR #30392:
URL: https://github.com/apache/doris/pull/30392#discussion_r1470762414
##########
be/src/vec/functions/function_string.h:
##########
@@ -1866,6 +1866,159 @@ class FunctionSubstringIndex : public IFunction {
String get_name() const override { return name; }
size_t get_number_of_arguments() const override { return 3; }
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ return std::make_shared<DataTypeString>();
+ }
+
+ bool use_default_implementation_for_nulls() const override { return true; }
Review Comment:
no need overrite the fun，the default is `true`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function)make SUBSTRING_INDEX DEPEND_ON_ARGUMENT [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1910787895
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 78b7553d7c0cf4ad323b8095d530f381f2e21dfa with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.3 seconds inserted 10000000 Rows, about 751K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1911522741
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit b7ecc51d500aaa808e669a22b2add2d0281bf8e2 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.4 seconds inserted 10000000 Rows, about 746K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](function)make SUBSTRING_INDEX DEPEND_ON_ARGUMENT [doris]
Posted by "koarz (via GitHub)" <gi...@apache.org>.
koarz commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1910705643
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #30392:
URL: https://github.com/apache/doris/pull/30392#discussion_r1467226512
##########
be/src/vec/functions/function_string.h:
##########
@@ -1866,6 +1866,159 @@ class FunctionSubstringIndex : public IFunction {
String get_name() const override { return name; }
size_t get_number_of_arguments() const override { return 3; }
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
Review Comment:
warning: method 'get_return_type_impl' can be made static [readability-convert-member-functions-to-static]
```suggestion
static DataTypePtr get_return_type_impl(const DataTypes& arguments) override {
```
##########
be/src/vec/functions/function_string.h:
##########
@@ -1866,6 +1866,159 @@
String get_name() const override { return name; }
size_t get_number_of_arguments() const override { return 3; }
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ return std::make_shared<DataTypeString>();
+ }
+
+ bool use_default_implementation_for_nulls() const override { return true; }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
Review Comment:
warning: function 'execute_impl' exceeds recommended size/complexity thresholds [readability-function-size]
```cpp
Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
^
```
<details>
<summary>Additional context</summary>
**be/src/vec/functions/function_string.h:1874:** 136 lines including whitespace and comments (threshold 80)
```cpp
Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
^
```
</details>
##########
be/src/vec/functions/function_string.h:
##########
@@ -1866,6 +1866,159 @@
String get_name() const override { return name; }
size_t get_number_of_arguments() const override { return 3; }
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ return std::make_shared<DataTypeString>();
+ }
+
+ bool use_default_implementation_for_nulls() const override { return true; }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
+ size_t result, size_t input_rows_count) const override {
Review Comment:
warning: method 'execute_impl' can be made static [readability-convert-member-functions-to-static]
```suggestion
static Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
size_t result, size_t input_rows_count) override {
```
##########
be/src/vec/functions/function_string.h:
##########
@@ -1866,6 +1866,159 @@
String get_name() const override { return name; }
size_t get_number_of_arguments() const override { return 3; }
+ DataTypePtr get_return_type_impl(const DataTypes& arguments) const override {
+ return std::make_shared<DataTypeString>();
+ }
+
+ bool use_default_implementation_for_nulls() const override { return true; }
+
+ Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
Review Comment:
warning: function 'execute_impl' has cognitive complexity of 86 (threshold 50) [readability-function-cognitive-complexity]
```cpp
Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& arguments,
^
```
<details>
<summary>Additional context</summary>
**be/src/vec/functions/function_string.h:1900:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (part_number == 0 || delimiter_size == 0) {
^
```
**be/src/vec/functions/function_string.h:1900:** +1
```cpp
if (part_number == 0 || delimiter_size == 0) {
^
```
**be/src/vec/functions/function_string.h:1901:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
for (size_t i = 0; i < input_rows_count; ++i) {
^
```
**be/src/vec/functions/function_string.h:1904:** +1, nesting level increased to 1
```cpp
} else if (part_number > 0) {
^
```
**be/src/vec/functions/function_string.h:1905:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
if (delimiter_size == 1) {
^
```
**be/src/vec/functions/function_string.h:1907:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
for (size_t i = 0; i < input_rows_count; ++i) {
^
```
**be/src/vec/functions/function_string.h:1911:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
while (num < part_number) {
^
```
**be/src/vec/functions/function_string.h:1915:** +5, including nesting penalty of 4, nesting level increased to 5
```cpp
if (pos != nullptr) {
^
```
**be/src/vec/functions/function_string.h:1918:** +1, nesting level increased to 5
```cpp
} else {
^
```
**be/src/vec/functions/function_string.h:1920:** +6, including nesting penalty of 5, nesting level increased to 6
```cpp
num = (num == 0) ? 0 : num + 1;
^
```
**be/src/vec/functions/function_string.h:1925:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
if (num == part_number) {
^
```
**be/src/vec/functions/function_string.h:1930:** +1, nesting level increased to 4
```cpp
} else {
^
```
**be/src/vec/functions/function_string.h:1935:** +1, nesting level increased to 2
```cpp
} else {
^
```
**be/src/vec/functions/function_string.h:1938:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
for (size_t i = 0; i < input_rows_count; ++i) {
^
```
**be/src/vec/functions/function_string.h:1942:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
while (num < part_number) {
^
```
**be/src/vec/functions/function_string.h:1946:** +5, including nesting penalty of 4, nesting level increased to 5
```cpp
if (pos < str.data + str.size) {
^
```
**be/src/vec/functions/function_string.h:1949:** +1, nesting level increased to 5
```cpp
} else {
^
```
**be/src/vec/functions/function_string.h:1951:** +6, including nesting penalty of 5, nesting level increased to 6
```cpp
num = (num == 0) ? 0 : num + 1;
^
```
**be/src/vec/functions/function_string.h:1956:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
if (num == part_number) {
^
```
**be/src/vec/functions/function_string.h:1961:** +1, nesting level increased to 4
```cpp
} else {
^
```
**be/src/vec/functions/function_string.h:1967:** +1, nesting level increased to 1
```cpp
} else {
^
```
**be/src/vec/functions/function_string.h:1970:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
for (size_t i = 0; i < input_rows_count; ++i) {
^
```
**be/src/vec/functions/function_string.h:1977:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
while (num <= part_number && offset >= 0) {
^
```
**be/src/vec/functions/function_string.h:1977:** +1
```cpp
while (num <= part_number && offset >= 0) {
^
```
**be/src/vec/functions/function_string.h:1979:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
if (offset != -1) {
^
```
**be/src/vec/functions/function_string.h:1980:** +5, including nesting penalty of 4, nesting level increased to 5
```cpp
if (++num == part_number) {
^
```
**be/src/vec/functions/function_string.h:1986:** +1, nesting level increased to 4
```cpp
} else {
^
```
**be/src/vec/functions/function_string.h:1990:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
num = (offset == -1 && num != 0) ? num + 1 : num;
^
```
**be/src/vec/functions/function_string.h:1990:** +1
```cpp
num = (offset == -1 && num != 0) ? num + 1 : num;
^
```
**be/src/vec/functions/function_string.h:1992:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
if (num == part_number) {
^
```
**be/src/vec/functions/function_string.h:1993:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
if (offset == -1) {
^
```
**be/src/vec/functions/function_string.h:1996:** +1, nesting level increased to 4
```cpp
} else {
^
```
**be/src/vec/functions/function_string.h:2002:** +1, nesting level increased to 3
```cpp
} else {
^
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1911508664
<details>
<summary>TPC-H: <b>Total hot run time: 38626 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b7ecc51d500aaa808e669a22b2add2d0281bf8e2, data reload: false
------ Round 1 ----------------------------------
q1 18632 5372 5536 5372
q2 2305 141 135 135
q3 11325 1168 1155 1155
q4 10734 752 772 752
q5 7751 3148 3176 3148
q6 203 121 124 121
q7 854 496 484 484
q8 9214 1887 1939 1887
q9 7198 6334 6349 6334
q10 8224 3020 3040 3020
q11 426 197 203 197
q12 362 188 191 188
q13 17988 3371 3346 3346
q14 245 217 211 211
q15 553 509 528 509
q16 431 376 382 376
q17 952 552 489 489
q18 7599 6987 6668 6668
q19 1553 1306 1290 1290
q20 597 308 284 284
q21 2799 2358 2407 2358
q22 362 302 310 302
Total cold run time: 110307 ms
Total hot run time: 38626 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5451 5229 5205 5205
q2 329 218 210 210
q3 3332 3220 3215 3215
q4 2075 2035 2046 2035
q5 5948 5884 6053 5884
q6 195 117 120 117
q7 2358 1918 1897 1897
q8 3199 3369 3377 3369
q9 9009 8700 8755 8700
q10 3890 3778 3887 3778
q11 550 432 461 432
q12 790 637 599 599
q13 16922 3190 3137 3137
q14 284 259 260 259
q15 561 504 512 504
q16 509 457 449 449
q17 1861 1832 1864 1832
q18 9674 20246 9653 9653
q19 23898 1543 1516 1516
q20 4574 1929 1919 1919
q21 14583 5426 5530 5426
q22 952 534 590 534
Total cold run time: 110944 ms
Total hot run time: 60670 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30392:
URL: https://github.com/apache/doris/pull/30392#issuecomment-1921685058
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org