You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "jacktengg (via GitHub)" <gi...@apache.org> on 2026/01/18 15:24:34 UTC
[PR] [improvement](cast) improve performance of casting string to decimal [doris]
jacktengg opened a new pull request, #60004:
URL: https://github.com/apache/doris/pull/60004
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3767509789
# BE Regression && UT Coverage Report
Increment line coverage `97.56% (40/41)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60004_e959fbc57871c9bb3a642d32790cae1a2c330c51_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60004_e959fbc57871c9bb3a642d32790cae1a2c330c51_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.41% (25434/35616) |
| Line Coverage | 53.95% (266181/493395) |
| Region Coverage | 51.70% (221246/427941) |
| Branch Coverage | 53.00% (94817/178897) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3766930627
<details>
<summary>TPC-DS: <b>Total hot run time: 174390 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e959fbc57871c9bb3a642d32790cae1a2c330c51, data reload: false
query5 4407 603 493 493
query6 330 244 227 227
query7 4219 454 263 263
query8 339 266 258 258
query9 8710 2917 2915 2915
query10 531 379 338 338
query11 15198 15200 14844 14844
query12 180 118 113 113
query13 1245 479 389 389
query14 6458 3106 2815 2815
query14_1 2664 2615 2645 2615
query15 190 187 172 172
query16 986 484 461 461
query17 1066 640 536 536
query18 2569 417 318 318
query19 216 218 193 193
query20 120 118 110 110
query21 212 138 112 112
query22 4046 3994 3920 3920
query23 16031 15641 15466 15466
query23_1 15630 15479 15399 15399
query24 7084 1602 1178 1178
query24_1 1153 1175 1188 1175
query25 499 425 401 401
query26 1257 266 162 162
query27 2775 450 286 286
query28 4558 2192 2166 2166
query29 752 546 448 448
query30 310 243 207 207
query31 802 633 588 588
query32 87 76 79 76
query33 553 342 313 313
query34 919 867 542 542
query35 728 754 681 681
query36 846 873 821 821
query37 133 95 84 84
query38 2737 2756 2654 2654
query39 763 743 745 743
query39_1 717 715 696 696
query40 219 140 125 125
query41 76 67 67 67
query42 108 103 111 103
query43 488 478 403 403
query44 1354 766 755 755
query45 186 193 183 183
query46 840 958 574 574
query47 1421 1431 1350 1350
query48 328 327 245 245
query49 617 438 347 347
query50 622 272 205 205
query51 3774 3796 3836 3796
query52 103 109 98 98
query53 300 331 278 278
query54 331 289 266 266
query55 83 80 78 78
query56 303 300 326 300
query57 1050 996 954 954
query58 294 272 264 264
query59 2022 2207 2035 2035
query60 351 341 316 316
query61 182 173 174 173
query62 414 351 315 315
query63 301 276 267 267
query64 4942 1309 946 946
query65 3726 3742 3692 3692
query66 1423 416 304 304
query67 15700 15568 15616 15568
query68 2626 1125 747 747
query69 444 360 330 330
query70 950 946 943 943
query71 314 306 294 294
query72 5389 3251 3233 3233
query73 605 723 308 308
query74 8744 8864 8563 8563
query75 2747 2807 2468 2468
query76 2393 1085 644 644
query77 353 385 325 325
query78 9781 9926 9182 9182
query79 1063 907 584 584
query80 1317 582 490 490
query81 544 263 235 235
query82 1342 144 111 111
query83 336 266 242 242
query84 251 118 92 92
query85 945 484 419 419
query86 412 296 329 296
query87 2835 2889 2722 2722
query88 3486 2598 2577 2577
query89 400 350 335 335
query90 1957 172 164 164
query91 168 151 134 134
query92 74 77 71 71
query93 958 915 558 558
query94 633 327 297 297
query95 591 332 373 332
query96 652 525 227 227
query97 2349 2404 2354 2354
query98 226 201 202 201
query99 582 601 508 508
Total cold run time: 248779 ms
Total hot run time: 174390 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3796829658
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3785252559
<details>
<summary>TPC-H: <b>Total hot run time: 31759 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 81180ec4e9e8e689bbd7337d117251dcd8bd3f20, data reload: false
------ Round 1 ----------------------------------
q1 17632 4787 4559 4559
q2 2038 304 193 193
q3 10234 1432 725 725
q4 10205 812 304 304
q5 7509 2175 1886 1886
q6 188 175 141 141
q7 932 730 594 594
q8 9283 1461 1149 1149
q9 5049 4613 4710 4613
q10 6821 1695 1278 1278
q11 522 299 299 299
q12 399 409 226 226
q13 17817 3828 3079 3079
q14 249 232 211 211
q15 618 531 535 531
q16 644 664 589 589
q17 665 821 511 511
q18 6757 6648 7047 6648
q19 1445 1001 671 671
q20 425 364 249 249
q21 3055 2302 2223 2223
q22 1107 1080 1100 1080
Total cold run time: 103594 ms
Total hot run time: 31759 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4982 5188 4940 4940
q2 342 416 336 336
q3 2376 2848 2424 2424
q4 1428 1859 1410 1410
q5 4415 4450 4421 4421
q6 228 170 124 124
q7 2080 1988 1841 1841
q8 2679 2386 2355 2355
q9 7256 7237 7220 7220
q10 2553 2657 2297 2297
q11 545 478 445 445
q12 695 752 638 638
q13 3575 4057 3087 3087
q14 270 288 271 271
q15 536 494 498 494
q16 614 665 604 604
q17 1089 1307 1268 1268
q18 7534 7234 7258 7234
q19 823 822 804 804
q20 1884 1961 1831 1831
q21 4496 4282 4195 4195
q22 1047 1058 985 985
Total cold run time: 51447 ms
Total hot run time: 49224 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3801634513
# BE Regression && UT Coverage Report
Increment line coverage `98.77% (80/81)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/60004_f24df0d0f7cd8c3af91a3190e34bf94b4825d29c_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/60004_f24df0d0f7cd8c3af91a3190e34bf94b4825d29c_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.43% (25541/35758) |
| Line Coverage | 54.04% (267829/495630) |
| Region Coverage | 51.62% (221930/429960) |
| Branch Coverage | 53.08% (95417/179761) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on code in PR #60004:
URL: https://github.com/apache/doris/pull/60004#discussion_r2729942130
##########
be/src/util/string_parser.hpp:
##########
@@ -107,6 +107,19 @@ inline const char* skip_leading_whitespace(const char* __restrict s, T& len) {
return s;
}
+// skip trailing ascii whitespaces,
+// return the pointer to the first char,
+// and update the len to the new length, which does not include
+// trailing whitespaces
+template <typename T>
+inline const char* skip_trailing_whitespaces(const char* s, T& len) {
Review Comment:
I'm afraid not, seems `skip_qualified_char` can only skip chars from begining, `skip_trailing_whitespaces` skips white spaces from end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3765405442
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3802488377
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3784693874
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3801232671
# BE UT Coverage Report
Increment line coverage `98.77% (80/81)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/f24df0d0f7cd8c3af91a3190e34bf94b4825d29c_f24df0d0f7cd8c3af91a3190e34bf94b4825d29c/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/f24df0d0f7cd8c3af91a3190e34bf94b4825d29c_f24df0d0f7cd8c3af91a3190e34bf94b4825d29c/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.48% (19144/36481) |
| Line Coverage | 35.87% (177963/496174) |
| Region Coverage | 32.30% (137410/425455) |
| Branch Coverage | 33.26% (59536/178993) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3799354467
<details>
<summary>TPC-H: <b>Total hot run time: 32410 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 070404164c2f81fe763ed228618d40f99e4dc78e, data reload: false
------ Round 1 ----------------------------------
q1 17643 5207 5056 5056
q2 2000 331 192 192
q3 10219 1330 732 732
q4 10208 819 312 312
q5 7566 2231 1856 1856
q6 236 183 148 148
q7 905 722 609 609
q8 9270 1421 1151 1151
q9 5518 4895 4745 4745
q10 6859 1952 1575 1575
q11 532 294 281 281
q12 390 373 221 221
q13 17794 4055 3253 3253
q14 239 240 212 212
q15 916 813 820 813
q16 666 677 623 623
q17 645 841 465 465
q18 7044 6495 6472 6472
q19 1469 1003 624 624
q20 399 344 229 229
q21 2694 1938 1873 1873
q22 1068 1032 968 968
Total cold run time: 104280 ms
Total hot run time: 32410 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5420 5302 5319 5302
q2 267 333 262 262
q3 2171 2658 2291 2291
q4 1332 1752 1313 1313
q5 4315 4224 4628 4224
q6 262 205 158 158
q7 2203 2010 1800 1800
q8 2606 2424 2456 2424
q9 7412 7572 7713 7572
q10 2870 2993 2688 2688
q11 546 474 448 448
q12 662 712 651 651
q13 3948 4314 3734 3734
q14 305 318 306 306
q15 888 856 792 792
q16 687 746 694 694
q17 1202 1423 1351 1351
q18 8330 7806 8170 7806
q19 871 817 853 817
q20 2063 2288 1960 1960
q21 4467 4256 3985 3985
q22 1063 1029 963 963
Total cold run time: 53890 ms
Total hot run time: 51541 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3766894694
<details>
<summary>TPC-H: <b>Total hot run time: 32120 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e959fbc57871c9bb3a642d32790cae1a2c330c51, data reload: false
------ Round 1 ----------------------------------
q1 17622 4231 4052 4052
q2 2105 349 242 242
q3 10125 1294 729 729
q4 10222 892 308 308
q5 7513 2094 1870 1870
q6 196 170 139 139
q7 946 788 643 643
q8 9263 1427 1229 1229
q9 4839 4562 4733 4562
q10 6738 1804 1399 1399
q11 506 295 282 282
q12 654 729 603 603
q13 17767 3902 3118 3118
q14 287 288 279 279
q15 594 509 516 509
q16 670 673 627 627
q17 657 807 562 562
q18 6714 6353 7002 6353
q19 1294 1085 673 673
q20 439 374 246 246
q21 3283 2698 2683 2683
q22 1164 1142 1012 1012
Total cold run time: 103598 ms
Total hot run time: 32120 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4528 4255 4399 4255
q2 341 409 329 329
q3 2263 2707 2377 2377
q4 1454 1823 1411 1411
q5 4493 4464 4353 4353
q6 220 173 130 130
q7 2071 1910 1786 1786
q8 2560 2590 2385 2385
q9 7222 7159 7140 7140
q10 2559 2654 2350 2350
q11 540 497 466 466
q12 686 757 611 611
q13 3709 4067 3116 3116
q14 273 287 262 262
q15 522 478 480 478
q16 630 635 595 595
q17 1098 1244 1247 1244
q18 7590 7374 7136 7136
q19 885 800 794 794
q20 1889 1962 1781 1781
q21 4534 4276 4150 4150
q22 1093 1043 986 986
Total cold run time: 51160 ms
Total hot run time: 48135 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3798696512
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3765406230
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3802733653
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3800312671
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on code in PR #60004:
URL: https://github.com/apache/doris/pull/60004#discussion_r2728276994
##########
be/src/util/string_parser.hpp:
##########
@@ -107,6 +107,19 @@ inline const char* skip_leading_whitespace(const char* __restrict s, T& len) {
return s;
}
+// skip trailing ascii whitespaces,
+// return the pointer to the first char,
+// and update the len to the new length, which does not include
+// trailing whitespaces
+template <typename T>
+inline const char* skip_trailing_whitespaces(const char* s, T& len) {
Review Comment:
can it be abstracted by `skip_qualified_char`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3799437868
<details>
<summary>ClickBench: <b>Total hot run time: 28.47 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 070404164c2f81fe763ed228618d40f99e4dc78e, data reload: false
query1 0.05 0.04 0.05
query2 0.09 0.05 0.05
query3 0.24 0.08 0.08
query4 1.61 0.12 0.11
query5 0.28 0.25 0.25
query6 1.17 0.67 0.67
query7 0.03 0.03 0.03
query8 0.05 0.04 0.04
query9 0.57 0.51 0.48
query10 0.56 0.53 0.56
query11 0.15 0.09 0.10
query12 0.14 0.10 0.11
query13 0.64 0.62 0.62
query14 1.08 1.08 1.04
query15 0.88 0.86 0.87
query16 0.40 0.40 0.40
query17 1.08 1.12 1.09
query18 0.23 0.21 0.21
query19 2.02 1.97 2.06
query20 0.02 0.01 0.01
query21 15.39 0.27 0.14
query22 5.22 0.04 0.04
query23 16.20 0.28 0.10
query24 1.67 0.67 0.51
query25 0.11 0.08 0.06
query26 0.13 0.14 0.13
query27 0.08 0.05 0.07
query28 4.53 1.14 0.96
query29 12.58 3.90 3.13
query30 0.28 0.13 0.12
query31 2.82 0.63 0.41
query32 3.22 0.60 0.49
query33 3.21 3.25 3.32
query34 16.72 5.39 4.74
query35 4.80 4.77 4.80
query36 0.66 0.50 0.50
query37 0.11 0.07 0.07
query38 0.06 0.04 0.04
query39 0.05 0.03 0.03
query40 0.19 0.17 0.16
query41 0.08 0.03 0.03
query42 0.04 0.03 0.03
query43 0.04 0.04 0.04
Total cold run time: 99.48 s
Total hot run time: 28.47 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3796829630
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3800055482
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3800818746
<details>
<summary>TPC-H: <b>Total hot run time: 32659 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f24df0d0f7cd8c3af91a3190e34bf94b4825d29c, data reload: false
------ Round 1 ----------------------------------
q1 17707 5216 5045 5045
q2 2055 314 192 192
q3 10190 1334 799 799
q4 10211 915 324 324
q5 7738 2175 1924 1924
q6 254 186 150 150
q7 883 729 627 627
q8 9269 1372 1061 1061
q9 5473 4817 4931 4817
q10 6814 1965 1584 1584
q11 510 293 280 280
q12 349 375 223 223
q13 17796 4068 3284 3284
q14 239 245 217 217
q15 914 833 810 810
q16 676 667 623 623
q17 639 838 474 474
q18 6920 6452 6476 6452
q19 1562 992 639 639
q20 398 353 233 233
q21 2630 2057 1951 1951
q22 1058 1009 950 950
Total cold run time: 104285 ms
Total hot run time: 32659 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5411 5317 5351 5317
q2 259 342 269 269
q3 2170 2690 2266 2266
q4 1347 1781 1287 1287
q5 4280 4155 4889 4155
q6 243 200 155 155
q7 2239 2015 1838 1838
q8 2555 2449 2404 2404
q9 7453 7697 7579 7579
q10 2878 3013 2644 2644
q11 553 479 450 450
q12 707 771 634 634
q13 3985 4462 3613 3613
q14 291 320 303 303
q15 918 849 811 811
q16 702 724 728 724
q17 1251 1388 1346 1346
q18 8295 7823 7757 7757
q19 842 872 833 833
q20 2106 2177 1942 1942
q21 4538 4235 4132 4132
q22 1054 1060 966 966
Total cold run time: 54077 ms
Total hot run time: 51425 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "jacktengg (via GitHub)" <gi...@apache.org>.
jacktengg commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3800256048
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3766782427
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3766948620
<details>
<summary>ClickBench: <b>Total hot run time: 27.05 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e959fbc57871c9bb3a642d32790cae1a2c330c51, data reload: false
query1 0.06 0.05 0.05
query2 0.10 0.05 0.04
query3 0.26 0.08 0.09
query4 1.60 0.11 0.12
query5 0.28 0.26 0.25
query6 1.16 0.66 0.65
query7 0.03 0.03 0.03
query8 0.05 0.05 0.04
query9 0.58 0.50 0.50
query10 0.55 0.55 0.53
query11 0.14 0.09 0.10
query12 0.14 0.11 0.10
query13 0.61 0.58 0.58
query14 0.96 0.94 0.94
query15 0.78 0.77 0.80
query16 0.39 0.40 0.40
query17 1.05 1.05 1.01
query18 0.24 0.21 0.21
query19 1.84 1.79 1.89
query20 0.02 0.01 0.02
query21 15.43 0.26 0.14
query22 5.34 0.05 0.05
query23 16.04 0.29 0.10
query24 0.92 0.77 0.62
query25 0.08 0.10 0.13
query26 0.14 0.13 0.12
query27 0.09 0.06 0.07
query28 4.32 1.10 0.89
query29 12.55 3.84 3.14
query30 0.28 0.14 0.11
query31 2.81 0.63 0.39
query32 3.23 0.56 0.45
query33 3.06 3.03 3.08
query34 16.04 5.12 4.43
query35 4.52 4.47 4.41
query36 0.66 0.50 0.49
query37 0.10 0.06 0.07
query38 0.07 0.04 0.04
query39 0.04 0.02 0.02
query40 0.17 0.15 0.13
query41 0.09 0.03 0.04
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 96.9 s
Total hot run time: 27.05 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3767075592
# BE UT Coverage Report
Increment line coverage `97.56% (40/41)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/e959fbc57871c9bb3a642d32790cae1a2c330c51_e959fbc57871c9bb3a642d32790cae1a2c330c51/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/e959fbc57871c9bb3a642d32790cae1a2c330c51_e959fbc57871c9bb3a642d32790cae1a2c330c51/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 52.43% (19053/36337) |
| Line Coverage | 35.72% (176439/493953) |
| Region Coverage | 32.27% (136627/423438) |
| Branch Coverage | 33.16% (59070/178136) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal [doris]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #60004:
URL: https://github.com/apache/doris/pull/60004#discussion_r2702569805
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -100,6 +203,141 @@ void read_file_to_vector_optimized(const std::string& filename, std::vector<std:
lines.push_back(std::move(line)); // Move instead of copy
}
}
+// Test cases
+/*
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128_tmp) {
+ {
+ // std::string s("1234567890.123456");
+ // std::string s("0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2147483647");
+ // std::string s("1234567890.123456e7");
+ // std::string s("1.5");
+ // std::string s("+0000.004e1");
+ // std::string s("+00099999999999999999999999999999999999.904e1");
+ std::string s("+000900000000000000000000000000000000000.004e1");
+ int type_precision = 38;
+ int type_scale = 1;
+ StringParser::ParseResult result = StringParser::PARSE_FAILURE;
+ auto value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ s.c_str(), s.size(), type_precision, type_scale, &result);
+ std::cout << "Parsed decimal128 value: " << value << ", ParseResult: " << result
+ << std::endl;
+ }
+}
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, false);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_nullable_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, true);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal64v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_18_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL64>(test_data_file, orig_line_count, 18,
+ 9, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal32v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_9_4.txt";
Review Comment:
The test methods contain hardcoded absolute file paths (e.g., "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_9_4.txt"). These paths are machine-specific and will fail on other development or CI environments. Consider using relative paths from the project root, or environment variables, or generate test data programmatically within the test itself.
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -100,6 +203,141 @@ void read_file_to_vector_optimized(const std::string& filename, std::vector<std:
lines.push_back(std::move(line)); // Move instead of copy
}
}
+// Test cases
+/*
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128_tmp) {
+ {
+ // std::string s("1234567890.123456");
+ // std::string s("0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2147483647");
+ // std::string s("1234567890.123456e7");
+ // std::string s("1.5");
+ // std::string s("+0000.004e1");
+ // std::string s("+00099999999999999999999999999999999999.904e1");
+ std::string s("+000900000000000000000000000000000000000.004e1");
+ int type_precision = 38;
+ int type_scale = 1;
+ StringParser::ParseResult result = StringParser::PARSE_FAILURE;
+ auto value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ s.c_str(), s.size(), type_precision, type_scale, &result);
+ std::cout << "Parsed decimal128 value: " << value << ", ParseResult: " << result
+ << std::endl;
+ }
+}
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
Review Comment:
The test methods contain hardcoded absolute file paths (e.g., "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt"). These paths are machine-specific and will fail on other development or CI environments. Consider using relative paths from the project root, or environment variables, or generate test data programmatically within the test itself.
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -27,6 +27,7 @@
#include "vec/columns/column_vector.h"
#include "vec/columns/common_column_test.h"
#include "vec/core/field.h"
+#include "vec/json/json_parser.h"
Review Comment:
The include "vec/json/json_parser.h" is not used anywhere in this file. Remove unused includes to keep the code clean and reduce compilation dependencies.
```suggestion
```
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -100,6 +203,141 @@ void read_file_to_vector_optimized(const std::string& filename, std::vector<std:
lines.push_back(std::move(line)); // Move instead of copy
}
}
+// Test cases
+/*
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128_tmp) {
+ {
+ // std::string s("1234567890.123456");
+ // std::string s("0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2147483647");
+ // std::string s("1234567890.123456e7");
+ // std::string s("1.5");
+ // std::string s("+0000.004e1");
+ // std::string s("+00099999999999999999999999999999999999.904e1");
+ std::string s("+000900000000000000000000000000000000000.004e1");
+ int type_precision = 38;
+ int type_scale = 1;
+ StringParser::ParseResult result = StringParser::PARSE_FAILURE;
+ auto value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ s.c_str(), s.size(), type_precision, type_scale, &result);
+ std::cout << "Parsed decimal128 value: " << value << ", ParseResult: " << result
+ << std::endl;
+ }
+}
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, false);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_nullable_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, true);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal64v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_18_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL64>(test_data_file, orig_line_count, 18,
+ 9, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal32v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_9_4.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL32>(test_data_file, orig_line_count, 9,
+ 4, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimalv2_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimalv2_test_data_27_9.txt";
Review Comment:
The test methods contain hardcoded absolute file paths (e.g., "/mnt/disk2/tengjianping/cast_perf/decimalv2_test_data_27_9.txt"). These paths are machine-specific and will fail on other development or CI environments. Consider using relative paths from the project root, or environment variables, or generate test data programmatically within the test itself.
##########
be/src/util/string_parser.cpp:
##########
@@ -160,51 +156,69 @@ typename PrimitiveTypeTraits<P>::CppType::NativeType StringParser::string_to_dec
// whose max value is std::numeric_limits<int32_t>::max() - 4,
// so int_part_count will be in range of int32_t,
// and int_part_count + exponent will be in range of int64_t
- int64_t tmp_actual_int_part_count = int_part_count + exponent;
- if (tmp_actual_int_part_count > std::numeric_limits<int>::max() ||
- tmp_actual_int_part_count < std::numeric_limits<int>::min()) {
- *result = StringParser::PARSE_OVERFLOW;
+ int64_t tmp_result_int_part_digit_count = int_part_count + exponent;
+ if (tmp_result_int_part_digit_count > std::numeric_limits<int>::max() ||
+ tmp_result_int_part_digit_count < std::numeric_limits<int>::min()) {
+ *result = is_negative ? StringParser::PARSE_UNDERFLOW : StringParser::PARSE_OVERFLOW;
return 0;
}
- int actual_int_part_count = tmp_actual_int_part_count;
+ int result_int_part_digit_count = tmp_result_int_part_digit_count;
int actual_frac_part_count = 0;
int digit_index = 0;
- if (actual_int_part_count >= 0) {
- int max_index = std::min(actual_int_part_count, total_digit_count);
+ if (result_int_part_digit_count >= 0) {
+ int max_index = std::min(found_dot ? (result_int_part_digit_count +
+ ((int_part_count > 0 && exponent > 0) ? 1 : 0))
+ : result_int_part_digit_count,
+ end_digit_index);
+ max_index = (max_index == std::numeric_limits<int>::min() ? end_digit_index : max_index);
// skip zero number
- for (; digit_index != max_index && digits[digit_index] == 0; ++digit_index) {
+ for (; digit_index != max_index && s[digit_index] == '0'; ++digit_index) {
}
// test 0.00, .00, 0.{00...}e2147483647
// 0.00000e2147483647
- if (max_index - digit_index > type_precision - type_scale) {
+ if (digit_index != max_index &&
+ (result_int_part_digit_count - digit_index > type_precision - type_scale)) {
*result = is_negative ? StringParser::PARSE_UNDERFLOW : StringParser::PARSE_OVERFLOW;
return 0;
}
// get int part number
for (; digit_index != max_index; ++digit_index) {
- int_part_number = int_part_number * 10 + digits[digit_index];
+ if (UNLIKELY(s[digit_index] == '.')) {
+ continue;
+ }
+ int_part_number = int_part_number * 10 + (s[digit_index] - '0');
}
- if (digit_index != actual_int_part_count) {
- int_part_number *= get_scale_multiplier<T>(actual_int_part_count - digit_index);
+ auto total_significant_digit_count = i - ((found_dot && int_part_count > 0) ? 1 : 0);
+ if (result_int_part_digit_count > total_significant_digit_count) {
+ int_part_number *= get_scale_multiplier<T>(result_int_part_digit_count -
+ total_significant_digit_count);
Review Comment:
The variable name 'total_significant_digit_count' is misleading. Based on the calculation 'i - ((found_dot && int_part_count > 0) ? 1 : 0)', this represents the total count of characters processed (including digits and potentially a decimal point), not just significant digits. Leading zeros were already skipped earlier in the function. A more accurate name would be 'total_digit_char_count' or 'digits_processed_count' to clarify what this value represents.
```suggestion
auto total_digit_char_count = i - ((found_dot && int_part_count > 0) ? 1 : 0);
if (result_int_part_digit_count > total_digit_char_count) {
int_part_number *= get_scale_multiplier<T>(result_int_part_digit_count -
total_digit_char_count);
```
##########
be/src/util/string_parser.cpp:
##########
@@ -109,6 +104,7 @@ typename PrimitiveTypeTraits<P>::CppType::NativeType StringParser::string_to_dec
}
// parse exponent if any
int64_t exponent = 0;
+ auto end_digit_index = i;
Review Comment:
The variable name 'end_digit_index' is misleading. This variable stores the index where digit parsing stopped (where 'e'/'E' was found or end of string), but it may include the position of a decimal point '.'. A more accurate name would be 'digit_section_end_index' or 'parse_end_index' to clarify that it marks the end of the numeric section before the exponent.
```suggestion
auto parse_end_index = i;
```
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -100,6 +203,141 @@ void read_file_to_vector_optimized(const std::string& filename, std::vector<std:
lines.push_back(std::move(line)); // Move instead of copy
}
}
+// Test cases
+/*
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128_tmp) {
+ {
+ // std::string s("1234567890.123456");
+ // std::string s("0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2147483647");
+ // std::string s("1234567890.123456e7");
+ // std::string s("1.5");
+ // std::string s("+0000.004e1");
+ // std::string s("+00099999999999999999999999999999999999.904e1");
+ std::string s("+000900000000000000000000000000000000000.004e1");
+ int type_precision = 38;
+ int type_scale = 1;
+ StringParser::ParseResult result = StringParser::PARSE_FAILURE;
+ auto value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ s.c_str(), s.size(), type_precision, type_scale, &result);
+ std::cout << "Parsed decimal128 value: " << value << ", ParseResult: " << result
+ << std::endl;
+ }
+}
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, false);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_nullable_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, true);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal64v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_18_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL64>(test_data_file, orig_line_count, 18,
+ 9, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal32v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_9_4.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL32>(test_data_file, orig_line_count, 9,
+ 4, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimalv2_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimalv2_test_data_27_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMALV2>(test_data_file, orig_line_count, 27,
+ 9, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_strict_mode_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000;
+
+ std::cout << "\n=== Testing with strict_mode=false ===\n";
+ perf_test_string_to_decimal_strict<PrimitiveType::TYPE_DECIMAL128I>(
+ test_data_file, orig_line_count, 38, 19, false, false);
+
+ std::cout << "\n=== Testing with strict_mode=true ===\n";
+ perf_test_string_to_decimal_strict<PrimitiveType::TYPE_DECIMAL128I>(
+ test_data_file, orig_line_count, 38, 19, false, true);
+}
+
+// Comparison test: String vs Direct decimal parsing
+TEST_F(FunctionCastToDecimalPerfTest, test_string_parse_comparison) {
+ std::vector<std::string> file_contents;
+ read_file_to_vector_optimized("/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt",
Review Comment:
The test methods contain hardcoded absolute file paths (e.g., "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt"). These paths are machine-specific and will fail on other development or CI environments. Consider using relative paths from the project root, or environment variables, or generate test data programmatically within the test itself.
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -100,6 +203,141 @@ void read_file_to_vector_optimized(const std::string& filename, std::vector<std:
lines.push_back(std::move(line)); // Move instead of copy
}
}
+// Test cases
+/*
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128_tmp) {
+ {
+ // std::string s("1234567890.123456");
+ // std::string s("0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2147483647");
+ // std::string s("1234567890.123456e7");
+ // std::string s("1.5");
+ // std::string s("+0000.004e1");
+ // std::string s("+00099999999999999999999999999999999999.904e1");
+ std::string s("+000900000000000000000000000000000000000.004e1");
+ int type_precision = 38;
+ int type_scale = 1;
+ StringParser::ParseResult result = StringParser::PARSE_FAILURE;
+ auto value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ s.c_str(), s.size(), type_precision, type_scale, &result);
+ std::cout << "Parsed decimal128 value: " << value << ", ParseResult: " << result
+ << std::endl;
+ }
+}
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, false);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_nullable_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, true);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal64v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_18_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL64>(test_data_file, orig_line_count, 18,
+ 9, false);
Review Comment:
Type mismatch in function call. The method 'perf_test_string_to_decimal' declared on line 81 expects a 'MutableColumnPtr' as the first parameter, but here it's being called with 'test_data_file' which is a 'std::string'. This code will fail to compile. Either the method signature needs an overload that accepts a string filename, or the test needs to load the data first into a column like the other test cases do.
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -100,6 +203,141 @@ void read_file_to_vector_optimized(const std::string& filename, std::vector<std:
lines.push_back(std::move(line)); // Move instead of copy
}
}
+// Test cases
+/*
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128_tmp) {
+ {
+ // std::string s("1234567890.123456");
+ // std::string s("0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2147483647");
+ // std::string s("1234567890.123456e7");
+ // std::string s("1.5");
+ // std::string s("+0000.004e1");
+ // std::string s("+00099999999999999999999999999999999999.904e1");
+ std::string s("+000900000000000000000000000000000000000.004e1");
+ int type_precision = 38;
+ int type_scale = 1;
+ StringParser::ParseResult result = StringParser::PARSE_FAILURE;
+ auto value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ s.c_str(), s.size(), type_precision, type_scale, &result);
+ std::cout << "Parsed decimal128 value: " << value << ", ParseResult: " << result
+ << std::endl;
+ }
+}
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, false);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_nullable_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, true);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal64v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_18_9.txt";
Review Comment:
The test methods contain hardcoded absolute file paths (e.g., "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_18_9.txt"). These paths are machine-specific and will fail on other development or CI environments. Consider using relative paths from the project root, or environment variables, or generate test data programmatically within the test itself.
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -100,6 +203,141 @@ void read_file_to_vector_optimized(const std::string& filename, std::vector<std:
lines.push_back(std::move(line)); // Move instead of copy
}
}
+// Test cases
+/*
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128_tmp) {
+ {
+ // std::string s("1234567890.123456");
+ // std::string s("0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2147483647");
+ // std::string s("1234567890.123456e7");
+ // std::string s("1.5");
+ // std::string s("+0000.004e1");
+ // std::string s("+00099999999999999999999999999999999999.904e1");
+ std::string s("+000900000000000000000000000000000000000.004e1");
+ int type_precision = 38;
+ int type_scale = 1;
+ StringParser::ParseResult result = StringParser::PARSE_FAILURE;
+ auto value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ s.c_str(), s.size(), type_precision, type_scale, &result);
+ std::cout << "Parsed decimal128 value: " << value << ", ParseResult: " << result
+ << std::endl;
+ }
+}
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, false);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_nullable_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, true);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal64v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_18_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL64>(test_data_file, orig_line_count, 18,
+ 9, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal32v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_9_4.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL32>(test_data_file, orig_line_count, 9,
+ 4, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimalv2_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimalv2_test_data_27_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMALV2>(test_data_file, orig_line_count, 27,
+ 9, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_strict_mode_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000;
+
+ std::cout << "\n=== Testing with strict_mode=false ===\n";
+ perf_test_string_to_decimal_strict<PrimitiveType::TYPE_DECIMAL128I>(
+ test_data_file, orig_line_count, 38, 19, false, false);
+
+ std::cout << "\n=== Testing with strict_mode=true ===\n";
+ perf_test_string_to_decimal_strict<PrimitiveType::TYPE_DECIMAL128I>(
+ test_data_file, orig_line_count, 38, 19, false, true);
Review Comment:
Type mismatch in function call. The method 'perf_test_string_to_decimal' declared on line 81 expects a 'MutableColumnPtr' as the first parameter, but here it's being called with 'test_data_file' which is a 'std::string'. This code will fail to compile. Either the method signature needs an overload that accepts a string filename, or the test needs to load the data first into a column like the other test cases do.
```suggestion
// Create string column and load data from file
auto dt_from = std::make_shared<DataTypeString>();
auto column_orig = dt_from->create_column();
{
MutableColumns columns;
columns.push_back(column_orig->get_ptr());
DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
EXPECT_EQ(column_orig->size(), orig_line_count);
}
perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL64>(column_orig, orig_line_count, 18, 9,
false);
}
TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal32v3_perf) {
std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_9_4.txt";
size_t orig_line_count = 10000000;
// Create string column and load data from file
auto dt_from = std::make_shared<DataTypeString>();
auto column_orig = dt_from->create_column();
{
MutableColumns columns;
columns.push_back(column_orig->get_ptr());
DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
EXPECT_EQ(column_orig->size(), orig_line_count);
}
perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL32>(column_orig, orig_line_count, 9, 4,
false);
}
TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimalv2_perf) {
std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimalv2_test_data_27_9.txt";
size_t orig_line_count = 10000000;
// Create string column and load data from file
auto dt_from = std::make_shared<DataTypeString>();
auto column_orig = dt_from->create_column();
{
MutableColumns columns;
columns.push_back(column_orig->get_ptr());
DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
EXPECT_EQ(column_orig->size(), orig_line_count);
}
perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMALV2>(column_orig, orig_line_count, 27, 9,
false);
}
TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_strict_mode_perf) {
std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
size_t orig_line_count = 10000000;
// Create string column and load data from file
auto dt_from = std::make_shared<DataTypeString>();
auto column_orig = dt_from->create_column();
{
MutableColumns columns;
columns.push_back(column_orig->get_ptr());
DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
EXPECT_EQ(column_orig->size(), orig_line_count);
}
std::cout << "\n=== Testing with strict_mode=false ===\n";
perf_test_string_to_decimal_strict<PrimitiveType::TYPE_DECIMAL128I>(
column_orig, orig_line_count, 38, 19, false, false);
std::cout << "\n=== Testing with strict_mode=true ===\n";
perf_test_string_to_decimal_strict<PrimitiveType::TYPE_DECIMAL128I>(
column_orig, orig_line_count, 38, 19, false, true);
```
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -100,6 +203,141 @@ void read_file_to_vector_optimized(const std::string& filename, std::vector<std:
lines.push_back(std::move(line)); // Move instead of copy
}
}
+// Test cases
+/*
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128_tmp) {
+ {
+ // std::string s("1234567890.123456");
+ // std::string s("0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2147483647");
+ // std::string s("1234567890.123456e7");
+ // std::string s("1.5");
+ // std::string s("+0000.004e1");
+ // std::string s("+00099999999999999999999999999999999999.904e1");
+ std::string s("+000900000000000000000000000000000000000.004e1");
+ int type_precision = 38;
+ int type_scale = 1;
+ StringParser::ParseResult result = StringParser::PARSE_FAILURE;
+ auto value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ s.c_str(), s.size(), type_precision, type_scale, &result);
+ std::cout << "Parsed decimal128 value: " << value << ", ParseResult: " << result
+ << std::endl;
+ }
+}
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, false);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_nullable_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, true);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal64v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_18_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL64>(test_data_file, orig_line_count, 18,
+ 9, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal32v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_9_4.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL32>(test_data_file, orig_line_count, 9,
+ 4, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimalv2_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimalv2_test_data_27_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMALV2>(test_data_file, orig_line_count, 27,
+ 9, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_strict_mode_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
Review Comment:
The test methods contain hardcoded absolute file paths (e.g., "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt"). These paths are machine-specific and will fail on other development or CI environments. Consider using relative paths from the project root, or environment variables, or generate test data programmatically within the test itself.
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -100,6 +203,141 @@ void read_file_to_vector_optimized(const std::string& filename, std::vector<std:
lines.push_back(std::move(line)); // Move instead of copy
}
}
+// Test cases
+/*
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128_tmp) {
+ {
+ // std::string s("1234567890.123456");
+ // std::string s("0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2147483647");
+ // std::string s("1234567890.123456e7");
+ // std::string s("1.5");
+ // std::string s("+0000.004e1");
+ // std::string s("+00099999999999999999999999999999999999.904e1");
+ std::string s("+000900000000000000000000000000000000000.004e1");
+ int type_precision = 38;
+ int type_scale = 1;
+ StringParser::ParseResult result = StringParser::PARSE_FAILURE;
+ auto value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ s.c_str(), s.size(), type_precision, type_scale, &result);
+ std::cout << "Parsed decimal128 value: " << value << ", ParseResult: " << result
+ << std::endl;
+ }
+}
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, false);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_nullable_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
Review Comment:
The test methods contain hardcoded absolute file paths (e.g., "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt"). These paths are machine-specific and will fail on other development or CI environments. Consider using relative paths from the project root, or environment variables, or generate test data programmatically within the test itself.
##########
be/src/util/string_parser.cpp:
##########
@@ -160,51 +156,69 @@ typename PrimitiveTypeTraits<P>::CppType::NativeType StringParser::string_to_dec
// whose max value is std::numeric_limits<int32_t>::max() - 4,
// so int_part_count will be in range of int32_t,
// and int_part_count + exponent will be in range of int64_t
- int64_t tmp_actual_int_part_count = int_part_count + exponent;
- if (tmp_actual_int_part_count > std::numeric_limits<int>::max() ||
- tmp_actual_int_part_count < std::numeric_limits<int>::min()) {
- *result = StringParser::PARSE_OVERFLOW;
+ int64_t tmp_result_int_part_digit_count = int_part_count + exponent;
+ if (tmp_result_int_part_digit_count > std::numeric_limits<int>::max() ||
+ tmp_result_int_part_digit_count < std::numeric_limits<int>::min()) {
+ *result = is_negative ? StringParser::PARSE_UNDERFLOW : StringParser::PARSE_OVERFLOW;
return 0;
}
- int actual_int_part_count = tmp_actual_int_part_count;
+ int result_int_part_digit_count = tmp_result_int_part_digit_count;
int actual_frac_part_count = 0;
int digit_index = 0;
- if (actual_int_part_count >= 0) {
- int max_index = std::min(actual_int_part_count, total_digit_count);
+ if (result_int_part_digit_count >= 0) {
+ int max_index = std::min(found_dot ? (result_int_part_digit_count +
+ ((int_part_count > 0 && exponent > 0) ? 1 : 0))
+ : result_int_part_digit_count,
+ end_digit_index);
Review Comment:
The calculation on lines 169-172 is complex and difficult to understand. The expression adds a conditional offset '((int_part_count > 0 && exponent > 0) ? 1 : 0)' when 'found_dot' is true, but the purpose of this offset is not immediately clear. Consider adding a comment explaining why this adjustment is needed and providing an example case where it applies (e.g., "Adjusts for the decimal point position when both integer digits and positive exponent exist").
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -100,6 +203,141 @@ void read_file_to_vector_optimized(const std::string& filename, std::vector<std:
lines.push_back(std::move(line)); // Move instead of copy
}
}
+// Test cases
+/*
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128_tmp) {
+ {
+ // std::string s("1234567890.123456");
+ // std::string s("0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2147483647");
+ // std::string s("1234567890.123456e7");
+ // std::string s("1.5");
+ // std::string s("+0000.004e1");
+ // std::string s("+00099999999999999999999999999999999999.904e1");
+ std::string s("+000900000000000000000000000000000000000.004e1");
+ int type_precision = 38;
+ int type_scale = 1;
+ StringParser::ParseResult result = StringParser::PARSE_FAILURE;
+ auto value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ s.c_str(), s.size(), type_precision, type_scale, &result);
+ std::cout << "Parsed decimal128 value: " << value << ", ParseResult: " << result
+ << std::endl;
+ }
+}
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, false);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_nullable_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, true);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal64v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_18_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL64>(test_data_file, orig_line_count, 18,
+ 9, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal32v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_9_4.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL32>(test_data_file, orig_line_count, 9,
+ 4, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimalv2_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimalv2_test_data_27_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMALV2>(test_data_file, orig_line_count, 27,
+ 9, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_strict_mode_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000;
+
+ std::cout << "\n=== Testing with strict_mode=false ===\n";
+ perf_test_string_to_decimal_strict<PrimitiveType::TYPE_DECIMAL128I>(
+ test_data_file, orig_line_count, 38, 19, false, false);
+
+ std::cout << "\n=== Testing with strict_mode=true ===\n";
+ perf_test_string_to_decimal_strict<PrimitiveType::TYPE_DECIMAL128I>(
+ test_data_file, orig_line_count, 38, 19, false, true);
+}
+
+// Comparison test: String vs Direct decimal parsing
+TEST_F(FunctionCastToDecimalPerfTest, test_string_parse_comparison) {
+ std::vector<std::string> file_contents;
+ read_file_to_vector_optimized("/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt",
+ file_contents);
+
+ size_t line_count = file_contents.size();
+ std::cout << "Read " << line_count << " lines" << std::endl;
+
+ // Test 1: Direct StringParser
+ int64_t duration_ns_parser = 0;
+ StringParser::ParseResult result;
+ {
+ SCOPED_RAW_TIMER(&duration_ns_parser);
+ for (size_t i = 0; i != line_count; ++i) {
+ auto decimal_value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ file_contents[i].c_str(), file_contents[i].size(), 38, 19, &result);
+ (void)decimal_value;
+ }
+ }
+ std::cout << "Direct StringParser time: "
+ << PrettyPrinter::print(duration_ns_parser, TUnit::TIME_NS) << "\n";
+
+ // Test 2: Cast function (already tested in other tests)
+ std::cout << "\nFor cast function performance, see test_string_to_decimal128v3_perf\n";
+}
+ */
Review Comment:
These test cases are commented out with a block comment (/* */). If these tests are no longer needed, they should be removed entirely. If they're intended for future use or debugging, consider using a feature flag or test attribute to disable them instead of commenting them out, which makes the codebase harder to maintain.
##########
be/src/util/string_parser.cpp:
##########
@@ -160,51 +156,69 @@ typename PrimitiveTypeTraits<P>::CppType::NativeType StringParser::string_to_dec
// whose max value is std::numeric_limits<int32_t>::max() - 4,
// so int_part_count will be in range of int32_t,
// and int_part_count + exponent will be in range of int64_t
- int64_t tmp_actual_int_part_count = int_part_count + exponent;
- if (tmp_actual_int_part_count > std::numeric_limits<int>::max() ||
- tmp_actual_int_part_count < std::numeric_limits<int>::min()) {
- *result = StringParser::PARSE_OVERFLOW;
+ int64_t tmp_result_int_part_digit_count = int_part_count + exponent;
+ if (tmp_result_int_part_digit_count > std::numeric_limits<int>::max() ||
+ tmp_result_int_part_digit_count < std::numeric_limits<int>::min()) {
+ *result = is_negative ? StringParser::PARSE_UNDERFLOW : StringParser::PARSE_OVERFLOW;
return 0;
}
- int actual_int_part_count = tmp_actual_int_part_count;
+ int result_int_part_digit_count = tmp_result_int_part_digit_count;
int actual_frac_part_count = 0;
int digit_index = 0;
- if (actual_int_part_count >= 0) {
- int max_index = std::min(actual_int_part_count, total_digit_count);
+ if (result_int_part_digit_count >= 0) {
+ int max_index = std::min(found_dot ? (result_int_part_digit_count +
+ ((int_part_count > 0 && exponent > 0) ? 1 : 0))
+ : result_int_part_digit_count,
+ end_digit_index);
+ max_index = (max_index == std::numeric_limits<int>::min() ? end_digit_index : max_index);
// skip zero number
- for (; digit_index != max_index && digits[digit_index] == 0; ++digit_index) {
+ for (; digit_index != max_index && s[digit_index] == '0'; ++digit_index) {
}
// test 0.00, .00, 0.{00...}e2147483647
// 0.00000e2147483647
- if (max_index - digit_index > type_precision - type_scale) {
+ if (digit_index != max_index &&
+ (result_int_part_digit_count - digit_index > type_precision - type_scale)) {
*result = is_negative ? StringParser::PARSE_UNDERFLOW : StringParser::PARSE_OVERFLOW;
return 0;
}
// get int part number
for (; digit_index != max_index; ++digit_index) {
- int_part_number = int_part_number * 10 + digits[digit_index];
+ if (UNLIKELY(s[digit_index] == '.')) {
+ continue;
+ }
+ int_part_number = int_part_number * 10 + (s[digit_index] - '0');
}
- if (digit_index != actual_int_part_count) {
- int_part_number *= get_scale_multiplier<T>(actual_int_part_count - digit_index);
+ auto total_significant_digit_count = i - ((found_dot && int_part_count > 0) ? 1 : 0);
Review Comment:
Incorrect variable used in calculation. At this point in the code, the variable 'i' has been modified during exponent parsing (see lines 111, 118, 126) and now points to the end of the string or past the exponent section. The calculation should use 'end_digit_index' instead of 'i', which correctly stores the position where the digit section ended (before the exponent). Using 'i' here will result in incorrect calculation of the total significant digit count.
```suggestion
auto total_significant_digit_count =
end_digit_index - ((found_dot && int_part_count > 0) ? 1 : 0);
```
##########
be/src/util/string_parser.cpp:
##########
@@ -160,51 +156,69 @@ typename PrimitiveTypeTraits<P>::CppType::NativeType StringParser::string_to_dec
// whose max value is std::numeric_limits<int32_t>::max() - 4,
// so int_part_count will be in range of int32_t,
// and int_part_count + exponent will be in range of int64_t
- int64_t tmp_actual_int_part_count = int_part_count + exponent;
- if (tmp_actual_int_part_count > std::numeric_limits<int>::max() ||
- tmp_actual_int_part_count < std::numeric_limits<int>::min()) {
- *result = StringParser::PARSE_OVERFLOW;
+ int64_t tmp_result_int_part_digit_count = int_part_count + exponent;
+ if (tmp_result_int_part_digit_count > std::numeric_limits<int>::max() ||
+ tmp_result_int_part_digit_count < std::numeric_limits<int>::min()) {
+ *result = is_negative ? StringParser::PARSE_UNDERFLOW : StringParser::PARSE_OVERFLOW;
return 0;
}
- int actual_int_part_count = tmp_actual_int_part_count;
+ int result_int_part_digit_count = tmp_result_int_part_digit_count;
int actual_frac_part_count = 0;
int digit_index = 0;
- if (actual_int_part_count >= 0) {
- int max_index = std::min(actual_int_part_count, total_digit_count);
+ if (result_int_part_digit_count >= 0) {
+ int max_index = std::min(found_dot ? (result_int_part_digit_count +
+ ((int_part_count > 0 && exponent > 0) ? 1 : 0))
+ : result_int_part_digit_count,
+ end_digit_index);
Review Comment:
The check for 'std::numeric_limits<int>::min()' seems suspicious. The condition checks if 'max_index == std::numeric_limits<int>::min()', which would only occur if both 'result_int_part_digit_count' and 'end_digit_index' are at their minimum values due to overflow in the ternary operation. This appears to be defensive coding for an edge case, but the logic is unclear. Consider adding a comment explaining when this condition would be true and why setting max_index to end_digit_index is the correct behavior in that case.
```suggestion
end_digit_index);
// Defensive check: under normal circumstances max_index is never INT_MIN, because
// result_int_part_digit_count is clamped to the range of int above and end_digit_index
// is derived from the parsed string length. If max_index were ever INT_MIN (e.g. due to
// unexpected overflow or undefined behavior in intermediate computations), fall back to
// using end_digit_index as a safe upper bound for digit iteration.
```
##########
be/test/vec/function/cast/cast_to_decimal128_perf.cpp:
##########
@@ -100,6 +203,141 @@ void read_file_to_vector_optimized(const std::string& filename, std::vector<std:
lines.push_back(std::move(line)); // Move instead of copy
}
}
+// Test cases
+/*
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128_tmp) {
+ {
+ // std::string s("1234567890.123456");
+ // std::string s("0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2147483647");
+ // std::string s("1234567890.123456e7");
+ // std::string s("1.5");
+ // std::string s("+0000.004e1");
+ // std::string s("+00099999999999999999999999999999999999.904e1");
+ std::string s("+000900000000000000000000000000000000000.004e1");
+ int type_precision = 38;
+ int type_scale = 1;
+ StringParser::ParseResult result = StringParser::PARSE_FAILURE;
+ auto value = StringParser::string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ s.c_str(), s.size(), type_precision, type_scale, &result);
+ std::cout << "Parsed decimal128 value: " << value << ", ParseResult: " << result
+ << std::endl;
+ }
+}
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, false);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal128v3_nullable_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_38_19.txt";
+ size_t orig_line_count = 10000000; // 10 million lines
+ // Create string column and load data from file
+ auto dt_from = std::make_shared<DataTypeString>();
+ auto column_orig = dt_from->create_column();
+
+ {
+ MutableColumns columns;
+ columns.push_back(column_orig->get_ptr());
+ DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
+ load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
+ EXPECT_EQ(column_orig->size(), orig_line_count);
+ }
+
+ int test_rounds = 10;
+ int64_t total_duration_ns = 0;
+ for (int i = 0; i < test_rounds; ++i) {
+ std::cout << "\n--- Test Round " << (i + 1) << " ---\n";
+ total_duration_ns += perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL128I>(
+ column_orig, orig_line_count, 38, 19, true);
+ }
+ auto avg_duration_ns = total_duration_ns / test_rounds;
+ std::cout << fmt::format("\nAverage time over {} rounds: {}\n", test_rounds,
+ PrettyPrinter::print(avg_duration_ns, TUnit::TIME_NS));
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal64v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_18_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL64>(test_data_file, orig_line_count, 18,
+ 9, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal32v3_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_9_4.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL32>(test_data_file, orig_line_count, 9,
+ 4, false);
+}
+
+TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimalv2_perf) {
+ std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimalv2_test_data_27_9.txt";
+ size_t orig_line_count = 10000000;
+ perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMALV2>(test_data_file, orig_line_count, 27,
Review Comment:
Type mismatch in function call. The method 'perf_test_string_to_decimal' declared on line 81 expects a 'MutableColumnPtr' as the first parameter, but here it's being called with 'test_data_file' which is a 'std::string'. This code will fail to compile. Either the method signature needs an overload that accepts a string filename, or the test needs to load the data first into a column like the other test cases do.
```suggestion
// Create string column and load data from file
auto dt_from = std::make_shared<DataTypeString>();
auto column_orig = dt_from->create_column();
{
MutableColumns columns;
columns.push_back(column_orig->get_ptr());
DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
EXPECT_EQ(column_orig->size(), orig_line_count);
}
perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL64>(column_orig, orig_line_count, 18,
9, false);
}
TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimal32v3_perf) {
std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimal_test_data_9_4.txt";
size_t orig_line_count = 10000000;
// Create string column and load data from file
auto dt_from = std::make_shared<DataTypeString>();
auto column_orig = dt_from->create_column();
{
MutableColumns columns;
columns.push_back(column_orig->get_ptr());
DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
EXPECT_EQ(column_orig->size(), orig_line_count);
}
perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMAL32>(column_orig, orig_line_count, 9,
4, false);
}
TEST_F(FunctionCastToDecimalPerfTest, test_string_to_decimalv2_perf) {
std::string test_data_file = "/mnt/disk2/tengjianping/cast_perf/decimalv2_test_data_27_9.txt";
size_t orig_line_count = 10000000;
// Create string column and load data from file
auto dt_from = std::make_shared<DataTypeString>();
auto column_orig = dt_from->create_column();
{
MutableColumns columns;
columns.push_back(column_orig->get_ptr());
DataTypeSerDeSPtrs serde = {dt_from->get_serde()};
load_columns_data_from_file(columns, serde, ';', {0}, test_data_file);
EXPECT_EQ(column_orig->size(), orig_line_count);
}
perf_test_string_to_decimal<PrimitiveType::TYPE_DECIMALV2>(column_orig, orig_line_count, 27,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3785304350
<details>
<summary>TPC-DS: <b>Total hot run time: 173183 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 81180ec4e9e8e689bbd7337d117251dcd8bd3f20, data reload: false
query5 4394 637 485 485
query6 330 212 200 200
query7 4220 474 270 270
query8 359 260 258 258
query9 8700 2909 2879 2879
query10 442 329 299 299
query11 15223 15069 14942 14942
query12 175 119 115 115
query13 1259 474 365 365
query14 5751 3054 2818 2818
query14_1 2698 2655 2699 2655
query15 212 200 175 175
query16 970 444 456 444
query17 1092 682 565 565
query18 2424 430 333 333
query19 192 176 155 155
query20 124 119 115 115
query21 220 144 120 120
query22 4153 4142 4293 4142
query23 16105 15522 15321 15321
query23_1 15432 15422 15432 15422
query24 7154 1534 1185 1185
query24_1 1166 1167 1177 1167
query25 532 453 413 413
query26 1258 264 155 155
query27 2781 453 281 281
query28 4597 2175 2153 2153
query29 792 544 462 462
query30 313 239 206 206
query31 759 632 564 564
query32 92 78 73 73
query33 526 385 330 330
query34 940 962 545 545
query35 719 766 678 678
query36 902 896 845 845
query37 140 101 94 94
query38 2743 2667 2651 2651
query39 793 746 735 735
query39_1 715 716 722 716
query40 232 137 130 130
query41 84 68 67 67
query42 98 92 98 92
query43 458 489 427 427
query44 1359 756 755 755
query45 190 185 180 180
query46 843 959 624 624
query47 1385 1491 1457 1457
query48 311 320 232 232
query49 604 441 339 339
query50 713 272 210 210
query51 3916 3797 3817 3797
query52 92 94 87 87
query53 211 237 162 162
query54 283 254 239 239
query55 79 74 74 74
query56 287 303 304 303
query57 997 1041 974 974
query58 267 257 259 257
query59 2085 2084 2081 2081
query60 337 329 301 301
query61 146 139 142 139
query62 397 353 316 316
query63 189 163 156 156
query64 4918 1147 824 824
query65 3832 3758 3788 3758
query66 1452 404 319 319
query67 15466 15500 15402 15402
query68 2411 1115 715 715
query69 383 308 287 287
query70 963 1012 903 903
query71 302 284 265 265
query72 5304 3357 3242 3242
query73 600 732 320 320
query74 8724 8770 8498 8498
query75 2256 2381 1911 1911
query76 2275 1055 659 659
query77 352 383 295 295
query78 9745 9956 9107 9107
query79 1060 906 575 575
query80 1305 518 429 429
query81 528 271 235 235
query82 1011 152 119 119
query83 327 261 242 242
query84 258 117 101 101
query85 869 458 409 409
query86 430 337 290 290
query87 2861 2862 2742 2742
query88 3504 2594 2559 2559
query89 306 270 243 243
query90 2036 172 151 151
query91 159 157 129 129
query92 71 71 70 70
query93 1107 1034 641 641
query94 634 331 307 307
query95 585 388 314 314
query96 648 498 227 227
query97 2362 2372 2306 2306
query98 208 207 206 206
query99 599 575 497 497
Total cold run time: 245467 ms
Total hot run time: 173183 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3785327720
<details>
<summary>ClickBench: <b>Total hot run time: 26.88 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 81180ec4e9e8e689bbd7337d117251dcd8bd3f20, data reload: false
query1 0.06 0.05 0.05
query2 0.10 0.05 0.05
query3 0.26 0.09 0.09
query4 1.60 0.12 0.11
query5 0.28 0.26 0.27
query6 1.15 0.66 0.66
query7 0.04 0.03 0.03
query8 0.06 0.04 0.05
query9 0.56 0.49 0.49
query10 0.56 0.55 0.54
query11 0.15 0.10 0.10
query12 0.14 0.10 0.11
query13 0.60 0.59 0.59
query14 0.97 0.95 0.92
query15 0.80 0.78 0.79
query16 0.39 0.42 0.40
query17 0.99 1.07 1.01
query18 0.22 0.21 0.22
query19 1.97 1.88 1.81
query20 0.02 0.01 0.02
query21 15.43 0.24 0.14
query22 4.92 0.06 0.05
query23 15.85 0.29 0.11
query24 1.48 0.36 0.57
query25 0.08 0.08 0.08
query26 0.14 0.13 0.14
query27 0.08 0.06 0.05
query28 4.17 1.07 0.89
query29 12.52 3.94 3.16
query30 0.31 0.17 0.13
query31 2.82 0.60 0.38
query32 3.24 0.54 0.46
query33 3.07 2.99 3.02
query34 15.81 5.07 4.42
query35 4.50 4.41 4.47
query36 0.65 0.50 0.49
query37 0.11 0.07 0.07
query38 0.08 0.05 0.04
query39 0.05 0.04 0.03
query40 0.17 0.15 0.14
query41 0.10 0.04 0.03
query42 0.05 0.04 0.03
query43 0.06 0.04 0.05
Total cold run time: 96.61 s
Total hot run time: 26.88 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #60004:
URL: https://github.com/apache/doris/pull/60004
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cast) improve performance of casting string to decimal and int [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #60004:
URL: https://github.com/apache/doris/pull/60004#issuecomment-3800890467
<details>
<summary>ClickBench: <b>Total hot run time: 28.25 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f24df0d0f7cd8c3af91a3190e34bf94b4825d29c, data reload: false
query1 0.06 0.04 0.05
query2 0.10 0.04 0.04
query3 0.26 0.08 0.08
query4 1.61 0.11 0.12
query5 0.28 0.25 0.26
query6 1.16 0.68 0.68
query7 0.03 0.02 0.03
query8 0.06 0.04 0.04
query9 0.57 0.50 0.48
query10 0.56 0.54 0.54
query11 0.14 0.09 0.10
query12 0.13 0.10 0.10
query13 0.64 0.62 0.62
query14 1.07 1.06 1.06
query15 0.88 0.86 0.86
query16 0.39 0.39 0.39
query17 1.10 1.14 1.11
query18 0.23 0.20 0.21
query19 2.08 2.05 2.02
query20 0.02 0.01 0.02
query21 15.44 0.26 0.15
query22 5.30 0.06 0.05
query23 16.29 0.28 0.10
query24 2.73 0.27 0.29
query25 0.09 0.11 0.09
query26 0.13 0.14 0.14
query27 0.10 0.05 0.05
query28 3.79 1.15 0.96
query29 12.56 3.93 3.12
query30 0.27 0.13 0.11
query31 2.82 0.65 0.40
query32 3.24 0.60 0.49
query33 3.29 3.22 3.23
query34 16.18 5.37 4.75
query35 4.82 4.75 4.80
query36 0.67 0.50 0.49
query37 0.11 0.06 0.06
query38 0.08 0.04 0.04
query39 0.04 0.03 0.03
query40 0.18 0.16 0.15
query41 0.09 0.03 0.04
query42 0.05 0.03 0.03
query43 0.05 0.04 0.03
Total cold run time: 99.69 s
Total hot run time: 28.25 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org