You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Hastyshell (via GitHub)" <gi...@apache.org> on 2025/12/22 03:35:44 UTC
[PR] [fix](compaction) Fix wrong JSON value used for tablet IDs in compaction status [doris]
Hastyshell opened a new pull request, #59236:
URL: https://github.com/apache/doris/pull/59236
### What problem does this PR solve?
Problem Summary:
```
thrift error, reason=THRIFT_EAGAIN (timed out)doris_be: /var/local/thirdparty/installed/include/rapidjson/document.h:1368: GenericValue<Encoding, Allocator> &rapidjson::GenericValue<rapidjson::UTF8<>>::AddMember(GenericValue<Encoding, Allocator> &, GenericValue<Encoding, Allocator> &, Allocator &) [Encoding = rapidjson::UTF8<>, Allocator = rapidjson::MemoryPoolAllocator<>]: Assertion `name.IsString()' failed.
13:43:12 *** Query id: 0-0 ***
13:43:12 *** is nereids: 0 ***
13:43:12 *** tablet id: 0 ***
13:43:12 *** Aborted at 1766294113 (unix time) try "date -d @1766294113" if you are using GNU date ***
13:43:12 *** Current BE git commitID: fa1f86a774 ***
13:43:12 *** SIGABRT unknown detail explain (@0x9b02) received by PID 39682 (TID 42141 OR 0x7b271cc0d700) from PID 39682; stack trace: ***
13:43:12 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/doris/be/src/common/signal_handler.h:420
13:43:12 1# 0x00007F305D563420 in /lib/x86_64-linux-gnu/libpthread.so.0
13:43:12 2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
13:43:12 3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
13:43:12 4# _nl_load_domain at /build/glibc-SzIz7B/glibc-2.31/intl/loadmsgcat.c:970
13:43:12 5# 0x00007F305D37EFD6 in /lib/x86_64-linux-gnu/libc.so.6
13:43:12 6# 0x000055C98F386B8A in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
13:43:12 7# doris::CompactionSubmitRegistry::jsonfy_compaction_status(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)::$_0::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::map<doris::DataDir*, std::unordered_set<std::shared_ptr<doris::Tablet>, std::hash<std::shared_ptr<doris::Tablet> >, std::equal_to<std::shared_ptr<doris::Tablet> >, std::allocator<std::shared_ptr<doris::Tablet> > >, std::less<doris::DataDir*>, std::allocator<std::pair<doris::DataDir* const, std::unordered_set<std::shared_ptr<doris::Tablet>, std::hash<std::shared_ptr<doris::Tablet> >, std::equal_to<std::shared_ptr<doris::Tablet> >, std::allocator<std::shared_ptr<doris::Tablet> > > > > > const&) const at /root/doris/be/src/olap/storage_engine.cpp:189
13:43:12 8# doris::CompactionSubmitRegistry::jsonfy_compaction_status(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) at /root/doris/be/src/olap/storage_engine.cpp:192
13:43:12 9# doris::CompactionAction::_handle_run_status_compaction(doris::HttpRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) at /root/doris/be/src/http/action/compaction_action.cpp:225
13:43:12 10# doris::CompactionAction::handle(doris::HttpRequest*) at /root/doris/be/src/http/action/compaction_action.cpp:385
13:43:12 11# 0x000055C9AC3CFEC7 in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
13:43:12 12# bufferevent_run_readcb_ at /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/doris-thirdparty-libevent-2.1.12.1/bufferevent.c:252
13:43:12 13# bufferevent_readcb at /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/doris-thirdparty-libevent-2.1.12.1/bufferevent_sock.c:227
13:43:12 14# event_process_active_single_queue at /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/doris-thirdparty-libevent-2.1.12.1/event.c:1682
13:43:12 15# event_process_active at /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/doris-thirdparty-libevent-2.1.12.1/event.c:1788
13:43:12 16# event_base_loop.constprop.0 at /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/doris-thirdparty-libevent-2.1.12.1/event.c:2006
13:43:12 17# std::_Function_handler<void (), doris::EvHttpServer::start()::$_0>::_M_invoke(std::_Any_data const&) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292
13:43:12 18# doris::ThreadPool::dispatch_thread() at /root/doris/be/src/util/threadpool.cpp:623
13:43:12 19# doris::Thread::supervise_thread(void*) at /root/doris/be/src/util/thread.cpp:461
13:43:12 20# asan_thread_start(void*) in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
13:43:12 21# start_thread at /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:478
```
Fix incorrect JSON value used for tablet IDs in compaction status
The compaction status JSON builder mistakenly pushed the compaction name
Value into the tablet ID array instead of using the tablet ID itself.
This resulted in incorrect JSON output and could trigger RapidJSON
assertions when the moved Value was later reused as an object key.
Fix the logic by:
- Using tablet ID as the array element instead of the compaction name
- Ensuring each rapidjson::Value is used only for its intended purpose
- Cleaning up JSON construction to avoid reusing moved values
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) Fix wrong JSON value used for tablet IDs in compaction status [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59236:
URL: https://github.com/apache/doris/pull/59236#issuecomment-3680848565
# BE UT Coverage Report
Increment line coverage `0.00% (0/10)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/aab8de20003895965100c3d5f7c858a7fc55dc9c_aab8de20003895965100c3d5f7c858a7fc55dc9c/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/aab8de20003895965100c3d5f7c858a7fc55dc9c_aab8de20003895965100c3d5f7c858a7fc55dc9c/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.41% (18923/35430) |
| Line Coverage | 39.28% (175377/446502) |
| Region Coverage | 33.84% (135725/401085) |
| Branch Coverage | 34.74% (58479/168348) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) Fix wrong JSON value used for tablet IDs in compaction status [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59236:
URL: https://github.com/apache/doris/pull/59236#issuecomment-3680266618
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) Fix wrong JSON value used for tablet IDs in compaction status [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59236:
URL: https://github.com/apache/doris/pull/59236#issuecomment-3680615684
<details>
<summary>TPC-DS: <b>Total hot run time: 179853 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aab8de20003895965100c3d5f7c858a7fc55dc9c, data reload: false
query5 4415 594 449 449
query6 348 240 220 220
query7 4214 482 276 276
query8 336 259 246 246
query9 8748 2563 2556 2556
query10 537 362 336 336
query11 15844 15163 14783 14783
query12 172 126 116 116
query13 1273 502 422 422
query14 5979 3018 2755 2755
query14_1 2684 2636 2739 2636
query15 214 203 180 180
query16 870 476 467 467
query17 1125 675 583 583
query18 2421 435 338 338
query19 238 225 196 196
query20 131 117 109 109
query21 220 147 116 116
query22 4012 3995 3969 3969
query23 16694 16262 16107 16107
query23_1 16063 16235 16024 16024
query24 7338 1637 1244 1244
query24_1 1251 1235 1282 1235
query25 527 479 452 452
query26 1267 272 158 158
query27 2739 470 306 306
query28 4500 2133 2134 2133
query29 816 570 505 505
query30 325 262 215 215
query31 819 707 640 640
query32 86 72 66 66
query33 546 341 309 309
query34 895 884 536 536
query35 770 786 696 696
query36 864 891 808 808
query37 135 96 78 78
query38 3006 3103 2911 2911
query39 741 748 706 706
query39_1 708 693 705 693
query40 222 142 123 123
query41 67 63 66 63
query42 112 106 110 106
query43 435 454 411 411
query44 1343 757 743 743
query45 196 192 183 183
query46 883 974 601 601
query47 1661 1730 1617 1617
query48 315 325 253 253
query49 667 428 346 346
query50 661 303 212 212
query51 3761 3796 3750 3750
query52 115 115 99 99
query53 329 359 297 297
query54 282 271 251 251
query55 83 85 79 79
query56 327 300 310 300
query57 1181 1151 1091 1091
query58 271 261 254 254
query59 2404 2411 2409 2409
query60 310 312 293 293
query61 168 168 161 161
query62 780 731 670 670
query63 328 299 303 299
query64 4986 1339 1016 1016
query65 4010 3970 3990 3970
query66 1467 445 322 322
query67 14901 15141 15105 15105
query68 2850 1035 769 769
query69 465 354 314 314
query70 1051 1015 960 960
query71 315 322 289 289
query72 6419 5148 5109 5109
query73 496 565 315 315
query74 8535 8967 8736 8736
query75 3129 3165 2898 2898
query76 2832 1139 729 729
query77 349 394 311 311
query78 9494 9777 8868 8868
query79 1174 833 626 626
query80 1286 650 555 555
query81 554 273 230 230
query82 1050 136 104 104
query83 328 257 240 240
query84 259 118 106 106
query85 929 532 471 471
query86 408 297 291 291
query87 3197 3204 3142 3142
query88 3371 2281 2257 2257
query89 471 417 405 405
query90 1978 163 161 161
query91 170 169 151 151
query92 73 68 60 60
query93 1083 911 559 559
query94 539 318 293 293
query95 589 381 318 318
query96 598 477 213 213
query97 2327 2298 2186 2186
query98 219 192 197 192
query99 1313 1339 1295 1295
Total cold run time: 253944 ms
Total hot run time: 179853 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) Fix wrong JSON value used for tablet IDs in compaction status [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59236:
URL: https://github.com/apache/doris/pull/59236#issuecomment-3685170877
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) Fix wrong JSON value used for tablet IDs in compaction status [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59236:
URL: https://github.com/apache/doris/pull/59236#issuecomment-3680595805
<details>
<summary>TPC-H: <b>Total hot run time: 36354 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aab8de20003895965100c3d5f7c858a7fc55dc9c, data reload: false
------ Round 1 ----------------------------------
q1 17654 4273 4061 4061
q2 2048 353 244 244
q3 10173 1338 750 750
q4 10208 851 315 315
q5 7663 2222 1893 1893
q6 224 166 135 135
q7 1018 870 709 709
q8 9369 1461 1188 1188
q9 7282 5339 5422 5339
q10 6873 2399 1992 1992
q11 528 335 299 299
q12 724 755 559 559
q13 17810 3689 3040 3040
q14 306 292 271 271
q15 596 518 510 510
q16 698 677 637 637
q17 689 844 474 474
q18 8307 7793 7803 7793
q19 1212 1033 666 666
q20 433 399 260 260
q21 4680 4247 4225 4225
q22 1155 1070 994 994
Total cold run time: 109650 ms
Total hot run time: 36354 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4305 4241 4264 4241
q2 358 429 331 331
q3 2562 2951 2467 2467
q4 1381 1889 1431 1431
q5 4532 4454 4438 4438
q6 212 164 126 126
q7 2040 1941 1953 1941
q8 2624 2546 2429 2429
q9 7590 7574 7099 7099
q10 2920 3134 2689 2689
q11 591 539 524 524
q12 632 704 575 575
q13 3309 3664 3047 3047
q14 265 280 259 259
q15 539 497 488 488
q16 627 660 598 598
q17 1108 1333 1395 1333
q18 7254 7059 7069 7059
q19 820 783 832 783
q20 1926 1974 1778 1778
q21 4671 4360 4186 4186
q22 1064 1067 974 974
Total cold run time: 51330 ms
Total hot run time: 48796 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) Fix wrong JSON value used for tablet IDs in compaction status [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59236:
URL: https://github.com/apache/doris/pull/59236#issuecomment-3685170957
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) Fix wrong JSON value used for tablet IDs in compaction status [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59236:
URL: https://github.com/apache/doris/pull/59236#issuecomment-3681169029
# BE Regression && UT Coverage Report
Increment line coverage `0.00% (0/10)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59236_aab8de20003895965100c3d5f7c858a7fc55dc9c_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59236_aab8de20003895965100c3d5f7c858a7fc55dc9c_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.29% (25035/34632) |
| Line Coverage | 59.05% (262902/445225) |
| Region Coverage | 53.98% (218759/405277) |
| Branch Coverage | 55.49% (93737/168926) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) Fix wrong JSON value used for tablet IDs in compaction status [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59236:
URL: https://github.com/apache/doris/pull/59236#issuecomment-3680626105
<details>
<summary>ClickBench: <b>Total hot run time: 27.42 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aab8de20003895965100c3d5f7c858a7fc55dc9c, data reload: false
query1 0.05 0.05 0.05
query2 0.09 0.05 0.05
query3 0.25 0.08 0.09
query4 1.60 0.11 0.10
query5 0.26 0.27 0.26
query6 1.17 0.63 0.64
query7 0.03 0.02 0.02
query8 0.05 0.04 0.04
query9 0.56 0.50 0.49
query10 0.55 0.55 0.55
query11 0.16 0.11 0.12
query12 0.15 0.12 0.11
query13 0.62 0.60 0.61
query14 1.02 0.99 0.99
query15 0.81 0.80 0.81
query16 0.43 0.41 0.42
query17 0.99 1.00 1.01
query18 0.24 0.21 0.22
query19 1.80 1.76 1.80
query20 0.02 0.01 0.01
query21 15.43 0.32 0.14
query22 4.82 0.05 0.04
query23 15.99 0.28 0.10
query24 1.40 1.44 0.34
query25 0.09 0.06 0.06
query26 0.14 0.13 0.13
query27 0.06 0.04 0.04
query28 3.40 1.19 1.03
query29 12.68 3.91 3.26
query30 0.28 0.14 0.12
query31 2.81 0.62 0.40
query32 3.22 0.54 0.45
query33 2.99 3.08 3.08
query34 16.93 5.23 4.55
query35 4.55 4.56 4.56
query36 0.67 0.50 0.49
query37 0.11 0.07 0.06
query38 0.07 0.04 0.04
query39 0.05 0.03 0.03
query40 0.17 0.14 0.14
query41 0.08 0.03 0.04
query42 0.04 0.03 0.03
query43 0.04 0.04 0.03
Total cold run time: 96.87 s
Total hot run time: 27.42 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) Fix wrong JSON value used for tablet IDs in compaction status [doris]
Posted by "Hastyshell (via GitHub)" <gi...@apache.org>.
Hastyshell commented on PR #59236:
URL: https://github.com/apache/doris/pull/59236#issuecomment-3680337698
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](compaction) Fix wrong JSON value used for tablet IDs in compaction status [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #59236:
URL: https://github.com/apache/doris/pull/59236
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org