You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "cambyzju (via GitHub)" <gi...@apache.org> on 2024/04/29 07:07:26 UTC
[PR] [fix](json) fix be crash while load json data [doris]
cambyzju opened a new pull request, #34283:
URL: https://github.com/apache/doris/pull/34283
## Proposed changes
Issue Number: close https://github.com/apache/doris/issues/34280
<!--Describe your changes.-->
BE crash while load json data with `fuzzy_parse` and `strip_outer_array` opened, and `simdjson` disabled.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](json) fix be crash while load json data [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang commented on PR #34283:
URL: https://github.com/apache/doris/pull/34283#issuecomment-2084669719
@cambyzju pls add a testcase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](json) fix be crash while load json data [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #34283:
URL: https://github.com/apache/doris/pull/34283#issuecomment-2082031811
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](json) fix be crash while load json data [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34283:
URL: https://github.com/apache/doris/pull/34283#issuecomment-2084261626
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](json) fix be crash while load json data [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34283:
URL: https://github.com/apache/doris/pull/34283#issuecomment-2084261643
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](json) fix be crash while load json data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34283:
URL: https://github.com/apache/doris/pull/34283#issuecomment-2082128889
<details>
<summary>TPC-DS: <b>Total hot run time: 184967 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b1da7f1903ae6545456de5889661a4f8c5050dd8, data reload: false
query1 909 364 355 355
query2 6447 2344 2202 2202
query3 6657 217 223 217
query4 24200 21280 21341 21280
query5 4167 420 428 420
query6 266 183 188 183
query7 4590 286 286 286
query8 249 194 193 193
query9 8656 2342 2317 2317
query10 457 242 249 242
query11 14666 14205 14416 14205
query12 145 92 88 88
query13 1659 368 370 368
query14 10046 8476 6812 6812
query15 261 194 169 169
query16 8106 259 257 257
query17 1780 567 547 547
query18 1997 284 262 262
query19 340 144 145 144
query20 96 84 83 83
query21 189 129 127 127
query22 5014 4829 4777 4777
query23 34054 33182 33237 33182
query24 10603 2967 2922 2922
query25 558 368 381 368
query26 687 151 143 143
query27 2128 315 312 312
query28 6020 2015 1993 1993
query29 845 599 592 592
query30 247 156 151 151
query31 942 724 744 724
query32 99 52 54 52
query33 697 243 244 243
query34 921 463 470 463
query35 813 660 661 660
query36 1065 887 873 873
query37 108 65 75 65
query38 3207 3010 2993 2993
query39 1564 1548 1567 1548
query40 208 125 129 125
query41 43 38 39 38
query42 107 100 98 98
query43 559 543 515 515
query44 1185 736 744 736
query45 293 257 251 251
query46 1065 706 712 706
query47 1938 1845 1835 1835
query48 371 295 284 284
query49 849 385 402 385
query50 767 368 380 368
query51 6962 6730 6762 6730
query52 98 91 92 91
query53 352 282 283 282
query54 307 241 251 241
query55 81 78 72 72
query56 246 214 217 214
query57 1202 1135 1111 1111
query58 220 199 196 196
query59 3357 3127 3203 3127
query60 276 232 233 232
query61 107 88 103 88
query62 625 440 433 433
query63 301 278 280 278
query64 8231 7106 7091 7091
query65 3116 3028 3107 3028
query66 784 335 346 335
query67 15221 14950 14846 14846
query68 5188 520 523 520
query69 473 298 299 298
query70 1178 1137 1104 1104
query71 407 272 265 265
query72 7460 2787 2412 2412
query73 703 316 322 316
query74 6492 6138 6034 6034
query75 3397 2705 2681 2681
query76 3318 1026 928 928
query77 386 269 271 269
query78 10886 10239 10196 10196
query79 5192 519 509 509
query80 2161 450 425 425
query81 537 223 212 212
query82 938 92 95 92
query83 302 168 170 168
query84 269 85 90 85
query85 1903 281 262 262
query86 468 314 315 314
query87 3253 3079 3070 3070
query88 5184 2323 2325 2323
query89 500 378 378 378
query90 2035 177 179 177
query91 128 96 94 94
query92 61 49 47 47
query93 5382 510 492 492
query94 1278 180 178 178
query95 401 299 307 299
query96 584 272 258 258
query97 3094 2977 2951 2951
query98 241 228 214 214
query99 1217 867 842 842
Total cold run time: 289342 ms
Total hot run time: 184967 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](json) fix be crash while load json data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34283:
URL: https://github.com/apache/doris/pull/34283#issuecomment-2082023621
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](json) fix be crash while load json data [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #34283:
URL: https://github.com/apache/doris/pull/34283#issuecomment-2082030671
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](json) fix be crash while load json data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34283:
URL: https://github.com/apache/doris/pull/34283#issuecomment-2082114043
TeamCity be ut coverage result:
Function Coverage: 35.60% (8932/25092)
Line Coverage: 27.22% (73657/270593)
Region Coverage: 26.41% (38047/144081)
Branch Coverage: 23.19% (19392/83640)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b1da7f1903ae6545456de5889661a4f8c5050dd8_b1da7f1903ae6545456de5889661a4f8c5050dd8/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](json) fix be crash while load json data [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #34283:
URL: https://github.com/apache/doris/pull/34283
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](json) fix be crash while load json data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #34283:
URL: https://github.com/apache/doris/pull/34283#issuecomment-2082109380
<details>
<summary>TPC-H: <b>Total hot run time: 41400 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b1da7f1903ae6545456de5889661a4f8c5050dd8, data reload: false
------ Round 1 ----------------------------------
q1 17610 4322 4228 4228
q2 2006 192 188 188
q3 10451 1243 1182 1182
q4 10204 757 858 757
q5 7546 2790 2787 2787
q6 215 127 133 127
q7 1058 643 618 618
q8 9214 2149 2154 2149
q9 9465 6879 6849 6849
q10 9586 3887 3902 3887
q11 457 260 238 238
q12 498 225 217 217
q13 18205 3167 3061 3061
q14 260 228 229 228
q15 533 471 484 471
q16 531 406 384 384
q17 999 723 755 723
q18 8391 7858 7725 7725
q19 5748 1592 1545 1545
q20 637 327 309 309
q21 5233 3447 4097 3447
q22 341 280 283 280
Total cold run time: 119188 ms
Total hot run time: 41400 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4537 4402 4419 4402
q2 359 279 275 275
q3 3189 2866 2997 2866
q4 1899 1611 1597 1597
q5 5485 5544 5516 5516
q6 214 123 124 123
q7 2373 1967 1965 1965
q8 3265 3451 3409 3409
q9 8880 8910 8953 8910
q10 4004 3728 3869 3728
q11 577 499 495 495
q12 796 630 630 630
q13 16009 3105 3089 3089
q14 310 298 281 281
q15 523 503 485 485
q16 499 452 428 428
q17 1808 1493 1521 1493
q18 7685 7629 7277 7277
q19 1635 1523 1545 1523
q20 1975 1773 1756 1756
q21 9980 4795 4636 4636
q22 543 507 487 487
Total cold run time: 76545 ms
Total hot run time: 55371 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org