You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "eldenmoon (via GitHub)" <gi...@apache.org> on 2025/10/10 02:58:52 UTC
[PR] [Fix](JsonFunction): Handle invalid escape sequences in JSON path to … [doris]
eldenmoon opened a new pull request, #56772:
URL: https://github.com/apache/doris/pull/56772
…prevent crash
The `get_json_token` function, which is used to parse JSON path expressions, could previously throw an unhandled
`boost::escaped_list_error` if the path string contained a malformed or invalid escape sequence (e.g., a trailing backslash).
This unhandled exception would cause the BE process to crash.Only in branch-3.0 and branch-2.1
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] branch-3.0 [Fix](JsonFunction) Handle invalid escape sequences in JSON path to prevent crash [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #56772:
URL: https://github.com/apache/doris/pull/56772#issuecomment-3388187109
<details>
<summary>TPC-DS: <b>Total hot run time: 193131 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 43af0b4d5608527dd41c5d0d0fb5abfb292af371, data reload: false
query1 987 393 393 393
query2 6350 1938 1880 1880
query3 8689 204 201 201
query4 33926 24033 23883 23883
query5 4265 474 449 449
query6 289 178 191 178
query7 4196 310 317 310
query8 281 220 236 220
query9 9399 2578 2571 2571
query10 484 273 262 262
query11 18391 15421 15236 15236
query12 160 103 102 102
query13 1535 442 424 424
query14 10221 7108 6632 6632
query15 233 173 182 173
query16 8047 509 492 492
query17 1585 619 604 604
query18 2204 321 315 315
query19 249 171 167 167
query20 124 122 122 122
query21 211 109 108 108
query22 4692 4482 4398 4398
query23 34846 34414 34622 34414
query24 11722 2965 2976 2965
query25 678 424 438 424
query26 1382 175 172 172
query27 2818 358 368 358
query28 7824 2153 2188 2153
query29 913 461 452 452
query30 268 158 164 158
query31 1064 837 854 837
query32 124 54 53 53
query33 773 296 301 296
query34 1001 524 531 524
query35 914 752 715 715
query36 1143 943 923 923
query37 126 67 70 67
query38 4091 3972 3922 3922
query39 1507 1529 1487 1487
query40 261 97 100 97
query41 51 48 49 48
query42 115 104 99 99
query43 545 494 488 488
query44 1348 851 841 841
query45 188 171 170 170
query46 1158 759 743 743
query47 2023 1940 1917 1917
query48 498 394 399 394
query49 904 428 403 403
query50 860 430 434 430
query51 7821 7340 7267 7267
query52 104 91 101 91
query53 270 183 187 183
query54 1117 468 473 468
query55 82 79 77 77
query56 264 240 250 240
query57 1345 1217 1187 1187
query58 240 214 209 209
query59 3225 3064 3105 3064
query60 282 256 268 256
query61 114 112 103 103
query62 888 714 701 701
query63 221 201 197 197
query64 4252 683 669 669
query65 3371 3270 3294 3270
query66 996 316 294 294
query67 15983 15854 15577 15577
query68 4808 579 609 579
query69 439 274 274 274
query70 1170 1112 1101 1101
query71 358 254 288 254
query72 6289 4048 4026 4026
query73 751 360 361 360
query74 10546 9345 9113 9113
query75 3384 2674 2647 2647
query76 2794 1101 1016 1016
query77 419 276 282 276
query78 10714 9715 9579 9579
query79 2013 611 605 605
query80 1079 425 425 425
query81 542 223 223 223
query82 606 86 88 86
query83 233 140 151 140
query84 237 79 83 79
query85 1366 306 287 287
query86 445 298 306 298
query87 4424 4295 4213 4213
query88 3892 2381 2351 2351
query89 414 291 290 290
query90 1954 186 185 185
query91 184 148 156 148
query92 66 50 51 50
query93 2524 553 550 550
query94 836 292 299 292
query95 371 258 254 254
query96 625 280 279 279
query97 3349 3148 3133 3133
query98 233 194 200 194
query99 1484 1305 1301 1301
Total cold run time: 304787 ms
Total hot run time: 193131 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] branch-3.0 [Fix](JsonFunction) Handle invalid escape sequences in JSON path to prevent crash [doris]
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon closed pull request #56772: branch-3.0 [Fix](JsonFunction) Handle invalid escape sequences in JSON path to prevent crash
URL: https://github.com/apache/doris/pull/56772
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] branch-3.0 [Fix](JsonFunction) Handle invalid escape sequences in JSON path to prevent crash [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #56772:
URL: https://github.com/apache/doris/pull/56772#issuecomment-3388172575
<details>
<summary>TPC-H: <b>Total hot run time: 39893 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 43af0b4d5608527dd41c5d0d0fb5abfb292af371, data reload: false
------ Round 1 ----------------------------------
q1 17944 6897 6708 6708
q2 2055 194 171 171
q3 10619 1109 1156 1109
q4 10416 772 732 732
q5 7712 2931 2826 2826
q6 211 135 135 135
q7 983 635 622 622
q8 9340 1978 2019 1978
q9 6656 6419 6409 6409
q10 7030 2242 2318 2242
q11 453 260 248 248
q12 400 220 223 220
q13 17800 3022 2976 2976
q14 243 204 211 204
q15 520 474 451 451
q16 483 374 381 374
q17 971 553 541 541
q18 7153 6553 6607 6553
q19 1423 1098 1066 1066
q20 455 198 204 198
q21 3856 3135 3226 3135
q22 1099 1019 995 995
Total cold run time: 107822 ms
Total hot run time: 39893 ms
----- Round 2, with runtime_filter_mode=off -----
q1 6650 6562 6582 6562
q2 338 232 240 232
q3 2942 2972 2918 2918
q4 2028 1850 1878 1850
q5 5660 5754 5705 5705
q6 217 133 131 131
q7 2284 1833 1837 1833
q8 3404 3520 3567 3520
q9 8812 8971 8862 8862
q10 3596 3500 3528 3500
q11 612 500 510 500
q12 838 593 596 593
q13 7545 3162 3200 3162
q14 307 282 273 273
q15 525 455 479 455
q16 496 454 448 448
q17 1835 1640 1610 1610
q18 8172 7681 7805 7681
q19 1671 1587 1474 1474
q20 2061 1896 1885 1885
q21 5162 5038 4943 4943
q22 1161 1062 1034 1034
Total cold run time: 66316 ms
Total hot run time: 59171 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] branch-3.0 [Fix](JsonFunction) Handle invalid escape sequences in JSON path to prevent crash [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #56772:
URL: https://github.com/apache/doris/pull/56772#issuecomment-3388189709
# BE UT Coverage Report
Increment line coverage `0.00% (0/21)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/43af0b4d5608527dd41c5d0d0fb5abfb292af371_43af0b4d5608527dd41c5d0d0fb5abfb292af371/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/43af0b4d5608527dd41c5d0d0fb5abfb292af371_43af0b4d5608527dd41c5d0d0fb5abfb292af371/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 42.10% (11245/26712) |
| Line Coverage | 32.63% (96288/295091) |
| Region Coverage | 30.55% (55269/180899) |
| Branch Coverage | 26.88% (27361/101784) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] branch-3.0 [Fix](JsonFunction): Handle invalid escape sequences in JSON path to prevent crash [doris]
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon commented on PR #56772:
URL: https://github.com/apache/doris/pull/56772#issuecomment-3388106206
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] branch-3.0 [Fix](JsonFunction) Handle invalid escape sequences in JSON path to prevent crash [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #56772:
URL: https://github.com/apache/doris/pull/56772#issuecomment-3388106646
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] branch-3.0 [Fix](JsonFunction) Handle invalid escape sequences in JSON path to prevent crash [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #56772:
URL: https://github.com/apache/doris/pull/56772#issuecomment-3388193492
<details>
<summary>ClickBench: <b>Total hot run time: 30.4 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 43af0b4d5608527dd41c5d0d0fb5abfb292af371, data reload: false
query1 0.03 0.03 0.03
query2 0.07 0.03 0.03
query3 0.24 0.06 0.06
query4 1.62 0.10 0.11
query5 0.52 0.51 0.50
query6 1.12 0.73 0.73
query7 0.03 0.02 0.01
query8 0.04 0.04 0.03
query9 0.56 0.50 0.51
query10 0.55 0.54 0.55
query11 0.14 0.10 0.11
query12 0.14 0.11 0.11
query13 0.60 0.60 0.59
query14 0.78 0.83 0.78
query15 0.85 0.83 0.83
query16 0.40 0.40 0.39
query17 1.06 1.05 1.06
query18 0.26 0.23 0.21
query19 1.94 1.88 1.88
query20 0.02 0.01 0.01
query21 15.42 0.59 0.59
query22 2.31 2.66 1.71
query23 16.94 1.14 0.96
query24 3.18 1.43 1.09
query25 0.11 0.11 0.05
query26 0.59 0.13 0.14
query27 0.05 0.05 0.05
query28 9.95 0.49 0.51
query29 12.59 3.23 3.17
query30 0.25 0.05 0.05
query31 2.87 0.40 0.38
query32 3.23 0.46 0.45
query33 2.98 3.06 3.03
query34 17.25 4.49 4.56
query35 4.55 4.51 4.58
query36 0.65 0.47 0.48
query37 0.08 0.06 0.06
query38 0.05 0.03 0.03
query39 0.03 0.02 0.02
query40 0.16 0.12 0.13
query41 0.08 0.02 0.02
query42 0.03 0.02 0.02
query43 0.04 0.04 0.03
Total cold run time: 104.36 s
Total hot run time: 30.4 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org