You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yujun777 (via GitHub)" <gi...@apache.org> on 2024/06/18 03:25:01 UTC
[PR] [fix](race) fix access colocate group ids race [doris]
yujun777 opened a new pull request, #36444:
URL: https://github.com/apache/doris/pull/36444
```
2024-06-17 21:42:30,205 ERROR (colocate group clone checker|390) [Daemon.run():118] daemon thread got exception. name: colocate group clone checker
java.util.ConcurrentModificationException: null
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1597) ~[?:?]
at java.util.HashMap$EntryIterator.next(HashMap.java:1630) ~[?:?]
at java.util.HashMap$EntryIterator.next(HashMap.java:1628) ~[?:?]
at com.google.common.collect.AbstractMapBasedMultimap$KeySet$1.next(AbstractMapBasedMultimap.java:964) ~[guava-32.1.2-jre.jar:?]
at org.apache.doris.clone.ColocateTableCheckerAndBalancer.matchGroups(ColocateTableCheckerAndBalancer.java:482) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.clone.ColocateTableCheckerAndBalancer.runAfterCatalogReady(ColocateTableCheckerAndBalancer.java:339) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
```
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2174906973
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2176361373
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2174893328
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2174894782
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2174893270
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2175091014
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2174897897
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2175172460
<details>
<summary>TPC-H: <b>Total hot run time: 39826 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4d15f7e95e60e9e64416c301abbec0349fdc177f, data reload: false
------ Round 1 ----------------------------------
q1 17598 4475 4436 4436
q2 2020 184 196 184
q3 10448 1083 1161 1083
q4 10189 748 709 709
q5 7459 2654 2601 2601
q6 225 137 139 137
q7 972 601 592 592
q8 9228 2064 2121 2064
q9 9063 6456 6466 6456
q10 9039 3740 3750 3740
q11 457 247 230 230
q12 568 221 228 221
q13 17768 2978 2961 2961
q14 266 221 220 220
q15 513 483 464 464
q16 505 377 380 377
q17 974 637 745 637
q18 8004 7393 7377 7377
q19 7122 1439 1474 1439
q20 654 329 321 321
q21 5271 3244 3941 3244
q22 391 333 359 333
Total cold run time: 118734 ms
Total hot run time: 39826 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4412 4320 4240 4240
q2 360 268 263 263
q3 3030 2942 2956 2942
q4 1959 1712 1716 1712
q5 5551 5512 5473 5473
q6 219 131 132 131
q7 2207 1858 1805 1805
q8 3260 3425 3420 3420
q9 8696 8750 8749 8749
q10 4087 3796 3799 3796
q11 601 488 514 488
q12 807 667 641 641
q13 17319 3173 3198 3173
q14 312 278 272 272
q15 529 492 492 492
q16 511 425 433 425
q17 1800 1466 1514 1466
q18 8193 7855 7963 7855
q19 1815 1560 1695 1560
q20 3094 1917 1849 1849
q21 5810 4789 4805 4789
q22 638 539 579 539
Total cold run time: 75210 ms
Total hot run time: 56080 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2174896677
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #36444:
URL: https://github.com/apache/doris/pull/36444
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2174917293
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2175216307
<details>
<summary>TPC-DS: <b>Total hot run time: 171014 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4d15f7e95e60e9e64416c301abbec0349fdc177f, data reload: false
query1 925 380 372 372
query2 6468 2367 2347 2347
query3 6635 209 211 209
query4 18936 17400 17381 17381
query5 3714 485 462 462
query6 243 165 172 165
query7 4581 304 289 289
query8 313 318 290 290
query9 8647 2395 2409 2395
query10 562 291 271 271
query11 10623 10060 9969 9969
query12 120 90 84 84
query13 1640 380 365 365
query14 10095 6846 6964 6846
query15 240 202 185 185
query16 7188 268 271 268
query17 1400 563 512 512
query18 1898 276 285 276
query19 196 162 157 157
query20 90 90 87 87
query21 206 126 132 126
query22 4567 4173 4001 4001
query23 33489 33811 33868 33811
query24 11290 2847 2903 2847
query25 598 374 384 374
query26 732 156 152 152
query27 2374 330 330 330
query28 5993 2114 2151 2114
query29 899 632 641 632
query30 256 151 153 151
query31 970 753 768 753
query32 90 53 57 53
query33 768 293 291 291
query34 1066 487 482 482
query35 733 636 611 611
query36 1132 993 1013 993
query37 147 73 76 73
query38 2936 2818 2830 2818
query39 889 846 864 846
query40 202 130 128 128
query41 58 53 55 53
query42 113 99 102 99
query43 595 546 542 542
query44 1187 715 716 715
query45 196 167 168 167
query46 1071 730 757 730
query47 1840 1801 1771 1771
query48 375 302 306 302
query49 832 395 396 395
query50 766 395 396 395
query51 6798 6640 6705 6640
query52 105 89 90 89
query53 358 293 284 284
query54 879 445 437 437
query55 74 72 71 71
query56 293 250 260 250
query57 1095 1009 1032 1009
query58 262 245 255 245
query59 3457 3218 3180 3180
query60 287 267 261 261
query61 94 96 89 89
query62 616 446 442 442
query63 313 287 280 280
query64 8492 2229 1741 1741
query65 3223 3090 3128 3090
query66 844 342 342 342
query67 15456 15014 14950 14950
query68 5472 534 534 534
query69 573 440 421 421
query70 1154 1154 1154 1154
query71 468 279 268 268
query72 7253 4248 2643 2643
query73 764 319 318 318
query74 5853 5494 5479 5479
query75 3747 2662 2655 2655
query76 3164 915 917 915
query77 604 293 298 293
query78 10451 9776 9854 9776
query79 2452 513 506 506
query80 2259 453 448 448
query81 600 218 215 215
query82 1239 101 103 101
query83 312 169 160 160
query84 263 91 84 84
query85 1360 280 267 267
query86 471 304 313 304
query87 3313 3092 3073 3073
query88 3635 2330 2323 2323
query89 471 385 376 376
query90 1731 186 183 183
query91 132 99 163 99
query92 62 48 49 48
query93 2014 509 492 492
query94 1134 191 186 186
query95 392 308 306 306
query96 589 269 261 261
query97 3209 3044 3027 3027
query98 220 196 188 188
query99 1256 867 826 826
Total cold run time: 270147 ms
Total hot run time: 171014 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](race) fix access colocate group ids race [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36444:
URL: https://github.com/apache/doris/pull/36444#issuecomment-2175233897
<details>
<summary>ClickBench: <b>Total hot run time: 30.95 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4d15f7e95e60e9e64416c301abbec0349fdc177f, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.23 0.06 0.06
query4 1.66 0.07 0.06
query5 0.52 0.48 0.49
query6 1.12 0.74 0.72
query7 0.02 0.02 0.02
query8 0.05 0.04 0.04
query9 0.54 0.50 0.50
query10 0.54 0.56 0.54
query11 0.16 0.12 0.12
query12 0.15 0.12 0.12
query13 0.60 0.59 0.59
query14 0.78 0.79 0.78
query15 0.83 0.81 0.82
query16 0.36 0.36 0.38
query17 0.97 0.98 0.98
query18 0.22 0.26 0.23
query19 1.92 1.81 1.73
query20 0.01 0.01 0.02
query21 15.40 0.65 0.64
query22 4.10 6.62 2.35
query23 18.26 1.37 1.18
query24 2.12 0.21 0.21
query25 0.16 0.09 0.09
query26 0.26 0.17 0.17
query27 0.07 0.08 0.08
query28 13.22 1.03 1.00
query29 12.58 3.31 3.28
query30 0.26 0.06 0.06
query31 2.86 0.38 0.38
query32 3.26 0.47 0.48
query33 2.91 2.94 2.90
query34 17.08 4.42 4.44
query35 4.47 4.45 4.47
query36 0.66 0.47 0.46
query37 0.18 0.15 0.16
query38 0.15 0.14 0.15
query39 0.04 0.03 0.04
query40 0.18 0.17 0.15
query41 0.10 0.05 0.05
query42 0.06 0.05 0.04
query43 0.04 0.04 0.03
Total cold run time: 109.22 s
Total hot run time: 30.95 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org