You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "morrySnow (via GitHub)" <gi...@apache.org> on 2026/03/13 06:36:58 UTC
[PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
morrySnow opened a new pull request, #61304:
URL: https://github.com/apache/doris/pull/61304
Move getStringValueForQuery, getStringValueInComplexTypeForQuery, and getStringValueForStreamLoad logic from 17 Expr subclasses into a centralized ExprToStringValueVisitor, following the same visitor pattern as ExprToSqlVisitor and ExprToThriftVisitor.
New classes:
- ExprToStringValueVisitor: visitor with visit methods for DateLiteral, FloatLiteral, BoolLiteral, NullLiteral, DecimalLiteral, ArrayLiteral, MapLiteral, StructLiteral, CastExpr, StringLiteral, IPv4Literal, IPv6Literal, VarBinaryLiteral, PlaceHolderExpr, JsonLiteral, MaxLiteral
- StringValueContext: carries FormatOptions and mode flags (forStreamLoad, inComplexType)
Expr base class methods now delegate to the visitor. Subclass overrides are removed. Also fixes pre-existing local variable naming in DateLiteral static block (DATE_KEY_FORMATTER -> dateKeyFmt) and makes convertToString private (no external callers).
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4066780451
<details>
<summary>TPC-H: <b>Total hot run time: 27067 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 14ed8a7b543986e6425d5ba2d1edc6545c70795c, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17606 4472 4306 4306
q2 q3 10646 810 537 537
q4 4691 353 247 247
q5 7605 1205 1033 1033
q6 179 174 144 144
q7 805 839 667 667
q8 10067 1439 1324 1324
q9 5619 4760 4739 4739
q10 6314 1918 1656 1656
q11 483 261 247 247
q12 748 605 467 467
q13 18061 3017 2166 2166
q14 233 239 210 210
q15 q16 752 754 677 677
q17 723 850 443 443
q18 5989 5440 5306 5306
q19 1123 999 602 602
q20 542 499 381 381
q21 4426 1895 1646 1646
q22 392 329 269 269
Total cold run time: 97004 ms
Total hot run time: 27067 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4893 4620 4613 4613
q2 q3 3932 4411 3937 3937
q4 933 1227 817 817
q5 4076 4482 4419 4419
q6 180 175 147 147
q7 1737 1682 1572 1572
q8 2591 2736 2553 2553
q9 7533 7328 7376 7328
q10 3766 4064 3641 3641
q11 507 465 604 465
q12 557 671 452 452
q13 2762 3211 2342 2342
q14 303 317 275 275
q15 q16 717 789 709 709
q17 1192 1414 1372 1372
q18 7333 7065 6847 6847
q19 917 872 868 868
q20 2034 2159 1997 1997
q21 3961 3540 3409 3409
q22 453 417 390 390
Total cold run time: 50377 ms
Total hot run time: 48153 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4072624340
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4073713117
<details>
<summary>TPC-H: <b>Total hot run time: 26584 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 791ed62b5967a3d85e036b763024f661c38c633e, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17637 4411 4276 4276
q2 q3 10658 771 532 532
q4 4679 357 246 246
q5 7561 1212 1003 1003
q6 173 174 146 146
q7 797 845 677 677
q8 9303 1456 1326 1326
q9 4802 4697 4661 4661
q10 6243 1912 1629 1629
q11 467 261 228 228
q12 694 583 467 467
q13 18021 2967 2184 2184
q14 229 226 214 214
q15 q16 724 752 662 662
q17 726 835 433 433
q18 6053 5414 5222 5222
q19 1104 987 614 614
q20 529 489 374 374
q21 4486 1833 1400 1400
q22 347 290 331 290
Total cold run time: 95233 ms
Total hot run time: 26584 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4814 4686 4567 4567
q2 q3 3934 4339 3780 3780
q4 877 1198 765 765
q5 4068 4386 4334 4334
q6 188 177 144 144
q7 1763 1670 1538 1538
q8 2465 2709 2564 2564
q9 7573 7291 7481 7291
q10 3806 3975 3632 3632
q11 519 440 422 422
q12 506 593 460 460
q13 2870 3177 2392 2392
q14 282 294 276 276
q15 q16 707 764 709 709
q17 1143 1400 1376 1376
q18 7055 6747 6692 6692
q19 907 918 957 918
q20 2119 2162 1995 1995
q21 3998 3641 3362 3362
q22 458 460 388 388
Total cold run time: 50052 ms
Total hot run time: 47605 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4066583909
# FE UT Coverage Report
Increment line coverage ` 84.78% (117/138)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61304_14ed8a7b543986e6425d5ba2d1edc6545c70795c/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61304_14ed8a7b543986e6425d5ba2d1edc6545c70795c/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4073783084
<details>
<summary>TPC-DS: <b>Total hot run time: 168273 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 791ed62b5967a3d85e036b763024f661c38c633e, data reload: false
query5 4357 653 520 520
query6 344 241 225 225
query7 4213 485 270 270
query8 347 245 234 234
query9 8673 2729 2750 2729
query10 508 419 351 351
query11 6950 5096 4908 4908
query12 192 136 125 125
query13 1306 462 366 366
query14 5739 3756 3503 3503
query14_1 2852 2886 2809 2809
query15 202 197 172 172
query16 991 472 443 443
query17 891 729 640 640
query18 2460 452 368 368
query19 226 214 193 193
query20 142 132 130 130
query21 221 133 111 111
query22 13402 14093 14640 14093
query23 16294 15932 15623 15623
query23_1 15835 15612 15324 15324
query24 7251 1629 1231 1231
query24_1 1246 1212 1241 1212
query25 552 507 409 409
query26 1241 263 147 147
query27 2786 475 297 297
query28 4505 1862 1847 1847
query29 850 555 486 486
query30 292 219 189 189
query31 998 942 881 881
query32 89 74 71 71
query33 519 330 280 280
query34 922 881 506 506
query35 645 681 591 591
query36 1110 1173 940 940
query37 132 95 86 86
query38 2923 2919 2853 2853
query39 845 836 824 824
query39_1 800 782 806 782
query40 242 162 137 137
query41 75 74 58 58
query42 254 258 256 256
query43 241 258 220 220
query44
query45 197 192 189 189
query46 871 971 627 627
query47 2068 2145 2081 2081
query48 309 316 227 227
query49 635 459 384 384
query50 686 277 226 226
query51 4106 4033 4031 4031
query52 262 268 261 261
query53 291 337 283 283
query54 301 274 277 274
query55 90 85 87 85
query56 317 326 305 305
query57 1918 1841 1709 1709
query58 302 273 278 273
query59 2821 2928 2701 2701
query60 347 334 318 318
query61 157 154 154 154
query62 625 602 533 533
query63 310 280 288 280
query64 5113 1283 998 998
query65
query66 1464 463 365 365
query67 24188 24375 24112 24112
query68
query69 404 336 293 293
query70 972 974 959 959
query71 344 304 306 304
query72 2780 2788 2459 2459
query73 538 544 313 313
query74 9624 9630 9390 9390
query75 2852 2744 2464 2464
query76 2302 1026 683 683
query77 389 373 310 310
query78 10988 11277 10499 10499
query79 1080 820 592 592
query80 1504 636 544 544
query81 556 267 235 235
query82 1080 153 119 119
query83 362 259 247 247
query84 298 125 100 100
query85 1100 491 449 449
query86 430 296 287 287
query87 3147 3111 3048 3048
query88 3515 2669 2641 2641
query89 437 368 355 355
query90 1929 185 179 179
query91 169 173 160 160
query92 80 74 71 71
query93 941 860 498 498
query94 645 313 303 303
query95 593 333 324 324
query96 652 514 229 229
query97 2508 2472 2380 2380
query98 233 226 220 220
query99 1035 1001 927 927
Total cold run time: 249148 ms
Total hot run time: 168273 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4074246454
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow merged PR #61304:
URL: https://github.com/apache/doris/pull/61304
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4066841180
<details>
<summary>TPC-DS: <b>Total hot run time: 168318 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 14ed8a7b543986e6425d5ba2d1edc6545c70795c, data reload: false
query5 4318 645 502 502
query6 331 218 203 203
query7 4207 465 277 277
query8 352 260 235 235
query9 8727 2782 2770 2770
query10 521 392 333 333
query11 6928 5118 4846 4846
query12 181 133 126 126
query13 1282 461 347 347
query14 5723 3708 3474 3474
query14_1 2823 2832 2810 2810
query15 201 201 178 178
query16 985 488 438 438
query17 1028 735 613 613
query18 2452 463 358 358
query19 219 211 185 185
query20 138 127 127 127
query21 215 136 117 117
query22 13254 14057 14829 14057
query23 16279 15887 15698 15698
query23_1 16248 15728 15699 15699
query24 7165 1604 1228 1228
query24_1 1231 1218 1220 1218
query25 531 495 399 399
query26 1247 260 148 148
query27 2784 502 292 292
query28 4448 1881 1876 1876
query29 830 562 480 480
query30 300 226 189 189
query31 1015 930 882 882
query32 85 71 67 67
query33 515 331 287 287
query34 907 877 526 526
query35 647 687 588 588
query36 1111 1163 961 961
query37 136 97 81 81
query38 2948 2966 2788 2788
query39 878 830 808 808
query39_1 803 789 801 789
query40 232 151 136 136
query41 62 58 62 58
query42 264 256 255 255
query43 249 248 227 227
query44
query45 201 190 182 182
query46 879 997 600 600
query47 2765 2141 2029 2029
query48 307 315 229 229
query49 627 458 366 366
query50 676 288 210 210
query51 4089 4023 4057 4023
query52 263 268 257 257
query53 284 346 281 281
query54 292 276 264 264
query55 92 89 83 83
query56 329 314 320 314
query57 1930 1641 1645 1641
query58 286 266 262 262
query59 2760 2962 2741 2741
query60 338 348 331 331
query61 147 149 152 149
query62 631 594 541 541
query63 309 273 273 273
query64 5045 1275 976 976
query65
query66 1478 461 357 357
query67 24234 24256 24113 24113
query68
query69 418 323 279 279
query70 962 974 982 974
query71 344 303 301 301
query72 2852 2800 2543 2543
query73 548 556 327 327
query74 9642 9596 9430 9430
query75 2853 2730 2431 2431
query76 2297 1031 688 688
query77 358 367 297 297
query78 10963 11168 10457 10457
query79 2298 825 591 591
query80 1773 618 532 532
query81 546 264 228 228
query82 997 158 120 120
query83 334 284 242 242
query84 266 117 94 94
query85 893 470 434 434
query86 434 304 293 293
query87 3109 3107 3066 3066
query88 3568 2691 2682 2682
query89 420 370 345 345
query90 2053 181 171 171
query91 168 157 134 134
query92 77 76 70 70
query93 1013 826 495 495
query94 641 315 285 285
query95 596 401 321 321
query96 640 517 225 225
query97 2462 2477 2401 2401
query98 246 221 222 221
query99 1029 1002 889 889
Total cold run time: 252101 ms
Total hot run time: 168318 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4065519685
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4065673627
<details>
<summary>TPC-DS: <b>Total hot run time: 168119 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c54c9d86d41f90dc91ea94c14ad8c1c787004083, data reload: false
query5 4312 621 515 515
query6 330 227 216 216
query7 4204 462 266 266
query8 339 229 222 222
query9 8671 2734 2741 2734
query10 514 422 329 329
query11 6962 5083 4893 4893
query12 191 136 139 136
query13 1284 473 358 358
query14 5562 3669 3444 3444
query14_1 2793 2825 2830 2825
query15 206 191 171 171
query16 970 454 435 435
query17 860 700 596 596
query18 2428 427 337 337
query19 217 206 178 178
query20 135 131 125 125
query21 210 127 108 108
query22 13299 13925 15239 13925
query23 16094 15757 15407 15407
query23_1 15619 15982 15933 15933
query24 7925 1619 1226 1226
query24_1 1280 1236 1234 1234
query25 590 483 438 438
query26 1247 264 150 150
query27 2779 489 299 299
query28 4522 1870 1878 1870
query29 889 591 501 501
query30 300 228 196 196
query31 1029 958 888 888
query32 88 79 72 72
query33 585 345 298 298
query34 923 874 531 531
query35 642 694 590 590
query36 1160 1206 935 935
query37 140 96 85 85
query38 2930 2903 2837 2837
query39 885 839 811 811
query39_1 789 792 783 783
query40 238 158 138 138
query41 67 68 62 62
query42 261 257 255 255
query43 244 253 223 223
query44
query45 199 195 187 187
query46 881 985 624 624
query47 2106 2539 2015 2015
query48 321 327 233 233
query49 673 459 389 389
query50 673 285 218 218
query51 4041 4120 4029 4029
query52 262 268 253 253
query53 286 332 282 282
query54 291 267 260 260
query55 99 86 80 80
query56 311 315 307 307
query57 1920 1777 1554 1554
query58 282 274 267 267
query59 2778 3012 2728 2728
query60 335 335 309 309
query61 149 143 147 143
query62 631 585 538 538
query63 310 279 270 270
query64 5154 1243 1004 1004
query65
query66 1473 444 353 353
query67 24248 24263 24216 24216
query68
query69 402 306 292 292
query70 976 928 913 913
query71 335 312 307 307
query72 2721 2592 2411 2411
query73 533 546 316 316
query74 9682 9570 9401 9401
query75 2854 2740 2485 2485
query76 2304 1032 688 688
query77 362 398 304 304
query78 10880 10998 10482 10482
query79 3046 767 579 579
query80 1755 627 550 550
query81 545 259 221 221
query82 997 149 114 114
query83 334 259 244 244
query84 300 113 98 98
query85 919 488 426 426
query86 409 300 301 300
query87 3157 3142 3050 3050
query88 3563 2677 2649 2649
query89 427 361 346 346
query90 1996 200 177 177
query91 166 160 137 137
query92 73 77 71 71
query93 1205 846 513 513
query94 654 304 276 276
query95 579 332 387 332
query96 635 520 233 233
query97 2501 2513 2368 2368
query98 243 231 218 218
query99 1007 987 925 925
Total cold run time: 251198 ms
Total hot run time: 168119 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4053107053
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #61304:
URL: https://github.com/apache/doris/pull/61304#discussion_r2944252627
##########
fe/fe-core/src/main/java/org/apache/doris/analysis/ExprToStringValueVisitor.java:
##########
@@ -0,0 +1,254 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.StructType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.FeConstants;
+import org.apache.doris.common.FractionalFormat;
+import org.apache.doris.foundation.format.FormatOptions;
+import org.apache.doris.nereids.util.DateUtils;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.time.ZoneId;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Visitor that generates string values for Expr instances, handling different
+ * output modes (query, stream load) and complex type nesting.
+ *
+ * <p>This visitor extracts the logic previously in {@code getStringValueForQuery},
+ * {@code getStringValueInComplexTypeForQuery}, and {@code getStringValueForStreamLoad}
+ * from Expr subclasses, following the same visitor pattern as {@link ExprToSqlVisitor}.
+ */
+public class ExprToStringValueVisitor extends ExprVisitor<String, StringValueContext> {
+ private static final Logger LOG = LogManager.getLogger(ExprToStringValueVisitor.class);
+
+ public static final ExprToStringValueVisitor INSTANCE = new ExprToStringValueVisitor();
+
+ @Override
+ public String visit(Expr expr, StringValueContext ctx) {
+ return expr.getStringValue();
+ }
+
+ @Override
+ public String visitDateLiteral(DateLiteral expr, StringValueContext ctx) {
+ String value;
+ if (expr.getType().isTimeStampTz()) {
+ try {
+ ZoneId dorisZone = DateUtils.getTimeZone();
+ String offset = dorisZone.getRules().getOffset(java.time.Instant.now()).toString();
+ DateLiteral dateLiteral = new DateLiteral(expr.getStringValue(),
+ ScalarType.createDatetimeV2Type(((ScalarType) expr.getType()).getScalarScale()));
+ value = dateLiteral.getStringValue() + offset;
+ } catch (Exception e) {
+ LOG.warn("generate timestamptz({})'s string value for query failed. ",
+ expr.getStringValue(), e);
+ value = expr.getStringValue();
+ }
+ } else {
+ value = expr.getStringValue();
+ }
+ if (ctx.isInComplexType()) {
+ return wrapWithQuotes(value, ctx);
+ }
+ return value;
+ }
+
+ @Override
+ public String visitFloatLiteral(FloatLiteral expr, StringValueContext ctx) {
+ String value;
+ if (expr.getType() == Type.TIMEV2) {
+ String timeStr = expr.getStringValue();
+ value = timeStr.substring(1, timeStr.length() - 1);
+ } else {
+ double dValue = expr.getValue();
+ if (expr.getType() == Type.FLOAT) {
+ Float fValue = (float) dValue;
+ if (fValue.equals(Float.POSITIVE_INFINITY)) {
+ dValue = Double.POSITIVE_INFINITY;
+ }
+ if (fValue.equals(Float.NEGATIVE_INFINITY)) {
+ dValue = Double.NEGATIVE_INFINITY;
+ }
+ }
+ value = FractionalFormat.getFormatStringValue(dValue,
+ expr.getType() == Type.DOUBLE ? 16 : 7,
+ expr.getType() == Type.DOUBLE ? "%.15E" : "%.6E");
+ }
+ if (ctx.isInComplexType() && expr.getType() == Type.TIMEV2) {
+ return wrapWithQuotes(value, ctx);
+ }
+ return value;
+ }
+
+ @Override
+ public String visitBoolLiteral(BoolLiteral expr, StringValueContext ctx) {
+ FormatOptions options = ctx.getFormatOptions();
+ if (options.level > 0) {
+ return options.isBoolValueNum() ? expr.getStringValue() : (expr.getValue() ? "true" : "false");
+ }
+ return expr.getStringValue();
+ }
+
+ @Override
+ public String visitNullLiteral(NullLiteral expr, StringValueContext ctx) {
+ if (ctx.isForStreamLoad()) {
+ return FeConstants.null_string;
+ }
Review Comment:
**Behavioral difference:** When `NullLiteral` is a child of a complex type (e.g., `ArrayLiteral`, `MapLiteral`) during stream load, the precedence of checks here differs from the old code.
**Old behavior:**
1. `ArrayLiteral.getStringValueForStreamLoad(options)` was NOT overridden.
2. `Expr.getStringValueForStreamLoad` delegated to `getStringValueForQuery(options)`.
3. `ArrayLiteral.getStringValueForQuery` called `child.getStringValueInComplexTypeForQuery(options)` on each child.
4. `NullLiteral.getStringValueInComplexTypeForQuery` returned `options.getNullFormat()` → `"null"`.
**New behavior:**
1. `visitArrayLiteral` calls `child.accept(this, ctx.asComplexType())` where ctx has `forStreamLoad=true`.
2. `asComplexType()` creates `{forStreamLoad=true, inComplexType=true}`.
3. `visitNullLiteral` hits the `isForStreamLoad()` check FIRST → returns `FeConstants.null_string` (`"\N"`).
So null inside `[1, null]` in stream load goes from `"null"` → `"\N"`.
Suggested fix — swap the check order to match old behavior:
```java
public String visitNullLiteral(NullLiteral expr, StringValueContext ctx) {
if (ctx.isInComplexType()) {
return ctx.getFormatOptions().getNullFormat();
}
if (ctx.isForStreamLoad()) {
return FeConstants.null_string;
}
return null;
}
```
Or add a test confirming the new behavior is intentional.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4072179923
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4074246678
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4066073163
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4065629052
<details>
<summary>TPC-H: <b>Total hot run time: 27074 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c54c9d86d41f90dc91ea94c14ad8c1c787004083, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17601 4616 4324 4324
q2 q3 10649 782 514 514
q4 4680 352 252 252
q5 7551 1231 1031 1031
q6 176 182 153 153
q7 780 844 680 680
q8 9292 1471 1332 1332
q9 4851 4800 4712 4712
q10 6336 1891 1682 1682
q11 490 271 248 248
q12 786 590 485 485
q13 18042 2909 2175 2175
q14 236 231 218 218
q15 q16 759 725 672 672
q17 714 865 470 470
q18 6205 5292 5168 5168
q19 1307 978 641 641
q20 535 499 382 382
q21 4577 1868 1618 1618
q22 500 368 317 317
Total cold run time: 96067 ms
Total hot run time: 27074 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4796 4576 4731 4576
q2 q3 3874 4333 3894 3894
q4 922 1243 858 858
q5 4074 4467 4322 4322
q6 173 184 147 147
q7 1754 1688 1526 1526
q8 2520 2779 2627 2627
q9 7636 7421 7372 7372
q10 3744 3983 3624 3624
q11 516 434 432 432
q12 497 589 487 487
q13 2749 3209 2375 2375
q14 288 305 277 277
q15 q16 727 750 816 750
q17 1252 1403 1432 1403
q18 7234 6730 6582 6582
q19 905 928 940 928
q20 2086 2171 1991 1991
q21 4012 3581 3481 3481
q22 469 447 411 411
Total cold run time: 50228 ms
Total hot run time: 48063 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4065900318
# FE UT Coverage Report
Increment line coverage ` 87.31% (117/134)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61304_c54c9d86d41f90dc91ea94c14ad8c1c787004083/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61304_c54c9d86d41f90dc91ea94c14ad8c1c787004083/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](Expr) Extract getString* methods to ExprToStringValueVisitor [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #61304:
URL: https://github.com/apache/doris/pull/61304#issuecomment-4072692429
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org