You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/07/08 02:30:59 UTC
[PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
w41ter opened a new pull request, #37415:
URL: https://github.com/apache/doris/pull/37415
Generally speaking, as long as a rowset has a version, it can be considered not to be in a pending state. However, if the rowset was created through ingesting binlogs, it will have a version but should still be considered in a pending state because the ingesting txn has not yet been committed.
This PR updates the condition for determining the pending state. If a rowset is COMMITTED, the txn should be allowed to roll back even if a version exists.
Cherry-pick #36551
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219516909
<details>
<summary>ClickBench: <b>Total hot run time: 30.9 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 50eb965b72dd6379918639419f8bc86330d5a624, data reload: false
query1 0.02 0.03 0.02
query2 0.07 0.02 0.02
query3 0.25 0.05 0.05
query4 1.78 0.07 0.07
query5 0.54 0.53 0.53
query6 1.23 0.63 0.62
query7 0.01 0.01 0.01
query8 0.03 0.03 0.02
query9 0.52 0.48 0.49
query10 0.54 0.53 0.53
query11 0.13 0.09 0.08
query12 0.12 0.09 0.09
query13 0.62 0.63 0.62
query14 0.78 0.79 0.78
query15 0.78 0.76 0.75
query16 0.36 0.37 0.36
query17 1.01 1.04 1.03
query18 0.20 0.27 0.21
query19 1.90 1.83 1.85
query20 0.01 0.01 0.01
query21 15.50 0.54 0.55
query22 1.92 2.13 1.61
query23 17.42 1.00 1.10
query24 7.53 1.50 1.13
query25 0.36 0.11 0.05
query26 0.84 0.14 0.14
query27 0.04 0.05 0.04
query28 5.05 0.73 0.79
query29 12.62 2.25 2.36
query30 0.56 0.52 0.51
query31 2.80 0.39 0.36
query32 3.40 0.49 0.49
query33 3.09 3.08 3.06
query34 15.34 4.80 4.79
query35 4.86 4.83 4.88
query36 1.05 1.00 1.02
query37 0.06 0.04 0.05
query38 0.04 0.02 0.03
query39 0.02 0.01 0.01
query40 0.16 0.14 0.14
query41 0.07 0.02 0.02
query42 0.02 0.01 0.01
query43 0.02 0.02 0.02
Total cold run time: 103.67 s
Total hot run time: 30.9 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2212865404
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219769375
TeamCity be ut coverage result:
Function Coverage: 37.89% (8116/21420)
Line Coverage: 29.55% (66498/225009)
Region Coverage: 29.03% (34277/118093)
Branch Coverage: 24.89% (17606/70724)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8a9934f3ac734f2b06b4f108f294b87c817a0ef8_8a9934f3ac734f2b06b4f108f294b87c817a0ef8/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219796294
<details>
<summary>ClickBench: <b>Total hot run time: 30.93 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8a9934f3ac734f2b06b4f108f294b87c817a0ef8, data reload: false
query1 0.02 0.03 0.02
query2 0.07 0.03 0.02
query3 0.25 0.04 0.04
query4 1.80 0.06 0.07
query5 0.54 0.52 0.51
query6 1.24 0.62 0.61
query7 0.01 0.00 0.01
query8 0.04 0.03 0.02
query9 0.53 0.49 0.48
query10 0.55 0.54 0.54
query11 0.12 0.08 0.09
query12 0.12 0.09 0.09
query13 0.62 0.61 0.61
query14 0.79 0.79 0.78
query15 0.79 0.76 0.77
query16 0.38 0.35 0.36
query17 1.01 0.99 1.01
query18 0.23 0.25 0.25
query19 1.93 1.88 1.87
query20 0.02 0.01 0.01
query21 15.48 0.57 0.55
query22 2.42 2.48 1.65
query23 17.41 0.97 0.86
query24 5.49 1.11 1.27
query25 0.37 0.10 0.05
query26 0.61 0.15 0.16
query27 0.04 0.04 0.04
query28 7.25 0.73 0.71
query29 12.67 2.43 2.42
query30 0.57 0.53 0.54
query31 2.81 0.39 0.38
query32 3.38 0.49 0.49
query33 3.12 3.11 3.06
query34 15.25 4.79 4.76
query35 4.84 4.84 4.82
query36 1.05 1.01 1.03
query37 0.06 0.05 0.04
query38 0.03 0.02 0.02
query39 0.02 0.01 0.02
query40 0.15 0.14 0.14
query41 0.07 0.01 0.01
query42 0.02 0.01 0.01
query43 0.02 0.02 0.02
Total cold run time: 104.19 s
Total hot run time: 30.93 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219764120
<details>
<summary>TPC-H: <b>Total hot run time: 50099 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8a9934f3ac734f2b06b4f108f294b87c817a0ef8, data reload: false
------ Round 1 ----------------------------------
q1 5082 4398 4391 4391
q2 759 201 152 152
q3 2099 1923 1924 1923
q4 2651 1317 1307 1307
q5 4022 3864 4051 3864
q6 236 124 132 124
q7 2063 1600 1640 1600
q8 2793 2754 2716 2716
q9 10679 10501 10277 10277
q10 5265 3547 3510 3510
q11 378 251 246 246
q12 462 305 300 300
q13 4490 4026 4056 4026
q14 364 326 336 326
q15 510 465 471 465
q16 645 578 591 578
q17 1116 957 976 957
q18 7248 6848 6931 6848
q19 1778 1657 1638 1638
q20 554 314 298 298
q21 4461 4134 4105 4105
q22 546 448 449 448
Total cold run time: 58201 ms
Total hot run time: 50099 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4338 4302 4281 4281
q2 316 223 232 223
q3 4195 4104 4166 4104
q4 2765 2734 2743 2734
q5 7217 7126 7109 7109
q6 237 121 119 119
q7 3249 2860 2770 2770
q8 4400 4489 4522 4489
q9 16882 16661 16769 16661
q10 4228 4251 4282 4251
q11 754 686 663 663
q12 1017 861 860 860
q13 4144 3777 3693 3693
q14 448 418 420 418
q15 515 446 462 446
q16 727 676 666 666
q17 3878 3863 3762 3762
q18 8792 8747 8745 8745
q19 1741 1748 1693 1693
q20 2348 2131 2092 2092
q21 8584 8452 8529 8452
q22 1141 1080 1058 1058
Total cold run time: 81916 ms
Total hot run time: 79289 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2213742662
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2212924431
<details>
<summary>ClickBench: <b>Total hot run time: 30.71 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f0b38d19132e14d28106e77bc0d182f0f1d95d11, data reload: false
query1 0.02 0.02 0.02
query2 0.07 0.03 0.02
query3 0.25 0.05 0.05
query4 1.77 0.07 0.07
query5 0.54 0.52 0.51
query6 1.23 0.62 0.62
query7 0.02 0.01 0.01
query8 0.04 0.03 0.03
query9 0.53 0.46 0.48
query10 0.54 0.53 0.53
query11 0.13 0.09 0.09
query12 0.11 0.09 0.09
query13 0.62 0.61 0.61
query14 0.79 0.79 0.78
query15 0.79 0.78 0.77
query16 0.38 0.39 0.40
query17 1.02 0.99 0.99
query18 0.23 0.27 0.23
query19 1.90 1.86 1.84
query20 0.01 0.02 0.01
query21 15.47 0.54 0.54
query22 2.00 2.29 1.96
query23 17.20 1.00 0.96
query24 8.38 0.60 0.96
query25 0.39 0.09 0.06
query26 0.81 0.15 0.16
query27 0.04 0.03 0.04
query28 5.27 0.77 0.77
query29 12.61 2.18 2.32
query30 0.57 0.58 0.53
query31 2.80 0.38 0.36
query32 3.39 0.50 0.48
query33 3.13 3.08 3.06
query34 15.64 4.81 4.80
query35 4.82 4.84 4.83
query36 1.06 1.02 1.02
query37 0.06 0.05 0.05
query38 0.04 0.03 0.02
query39 0.02 0.02 0.01
query40 0.16 0.14 0.14
query41 0.06 0.02 0.01
query42 0.02 0.02 0.02
query43 0.02 0.01 0.01
Total cold run time: 104.95 s
Total hot run time: 30.71 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2212903492
<details>
<summary>TPC-H: <b>Total hot run time: 50010 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f0b38d19132e14d28106e77bc0d182f0f1d95d11, data reload: false
------ Round 1 ----------------------------------
q1 17736 4453 4462 4453
q2 2106 157 151 151
q3 10432 1892 1919 1892
q4 10309 1255 1363 1255
q5 8613 3949 3923 3923
q6 232 127 148 127
q7 2055 1589 1557 1557
q8 9364 2759 2729 2729
q9 10732 10301 10193 10193
q10 8684 3556 3500 3500
q11 417 252 255 252
q12 482 303 307 303
q13 18347 3950 4045 3950
q14 373 318 332 318
q15 516 465 462 462
q16 667 574 565 565
q17 1144 979 967 967
q18 7356 6974 6839 6839
q19 1775 1689 1696 1689
q20 568 306 303 303
q21 4516 4138 4142 4138
q22 531 444 447 444
Total cold run time: 116955 ms
Total hot run time: 50010 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4337 4361 4279 4279
q2 326 226 228 226
q3 4185 4154 4154 4154
q4 2795 2768 2749 2749
q5 7136 7110 7094 7094
q6 241 122 119 119
q7 3254 2873 2829 2829
q8 4400 4510 4540 4510
q9 16960 16767 16850 16767
q10 4272 4300 4260 4260
q11 753 713 681 681
q12 1042 847 851 847
q13 7210 3729 3745 3729
q14 443 436 424 424
q15 513 444 462 444
q16 760 688 682 682
q17 3886 3887 3840 3840
q18 8845 8689 8840 8689
q19 1720 1712 1644 1644
q20 2361 2108 2075 2075
q21 8520 8426 8560 8426
q22 1036 989 971 971
Total cold run time: 84995 ms
Total hot run time: 79439 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219476430
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219733055
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #37415:
URL: https://github.com/apache/doris/pull/37415#discussion_r1667876925
##########
be/src/olap/rowset/rowset.h:
##########
@@ -164,6 +164,7 @@ class Rowset : public std::enable_shared_from_this<Rowset> {
int64_t newest_write_timestamp() const { return rowset_meta()->newest_write_timestamp(); }
bool is_segments_overlapping() const { return rowset_meta()->is_segments_overlapping(); }
KeysType keys_type() { return _schema->keys_type(); }
+ RowsetStatePB rowset_meta_state() const { return rowset_meta()->rowset_state(); }
Review Comment:
warning: function 'rowset_meta_state' should be marked [[nodiscard]] [modernize-use-nodiscard]
```suggestion
; }[[nodiscard]]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2212865194
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2212918797
<details>
<summary>TPC-DS: <b>Total hot run time: 206004 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f0b38d19132e14d28106e77bc0d182f0f1d95d11, data reload: false
query1 934 440 402 402
query2 6563 2840 2711 2711
query3 6925 206 203 203
query4 20226 18231 18075 18075
query5 19723 6650 6614 6614
query6 304 227 227 227
query7 4162 316 323 316
query8 437 421 431 421
query9 3209 2751 2676 2676
query10 454 324 302 302
query11 11310 10769 10710 10710
query12 127 82 78 78
query13 5608 707 687 687
query14 18133 14103 13832 13832
query15 367 261 251 251
query16 6471 293 269 269
query17 1725 1471 900 900
query18 2296 433 441 433
query19 219 154 155 154
query20 84 80 83 80
query21 197 103 99 99
query22 5330 5021 4926 4926
query23 33153 32209 32139 32139
query24 6941 6600 6573 6573
query25 542 432 431 431
query26 530 174 169 169
query27 1848 309 315 309
query28 6077 2394 2347 2347
query29 2884 2785 2759 2759
query30 249 177 175 175
query31 907 730 735 730
query32 64 65 63 63
query33 400 270 276 270
query34 870 485 498 485
query35 1119 917 949 917
query36 1283 1223 1110 1110
query37 87 63 61 61
query38 3121 2954 2991 2954
query39 1388 1363 1360 1360
query40 209 93 101 93
query41 49 44 44 44
query42 84 80 89 80
query43 823 733 662 662
query44 1141 718 727 718
query45 250 241 239 239
query46 1267 1009 1005 1005
query47 2028 1685 1760 1685
query48 1014 755 732 732
query49 637 378 375 375
query50 876 624 616 616
query51 4788 4669 4736 4669
query52 89 83 81 81
query53 474 333 327 327
query54 2718 2515 2526 2515
query55 97 79 80 79
query56 275 234 219 219
query57 1221 1121 1060 1060
query58 227 227 220 220
query59 4057 4141 3944 3944
query60 221 199 220 199
query61 98 98 96 96
query62 823 561 444 444
query63 501 348 342 342
query64 2607 1549 1445 1445
query65 3762 3667 3635 3635
query66 779 382 393 382
query67 17982 15738 15334 15334
query68 8653 644 657 644
query69 560 362 351 351
query70 1535 1467 1720 1467
query71 406 301 316 301
query72 6571 3546 3560 3546
query73 739 322 324 322
query74 6350 5897 5910 5897
query75 5020 3770 3806 3770
query76 5067 1209 1202 1202
query77 748 266 258 258
query78 12859 12122 11908 11908
query79 8834 654 674 654
query80 1345 407 413 407
query81 500 261 242 242
query82 1708 101 102 101
query83 175 141 132 132
query84 256 76 73 73
query85 943 334 332 332
query86 347 295 312 295
query87 3254 3070 3099 3070
query88 5161 2333 2336 2333
query89 392 286 311 286
query90 1891 216 209 209
query91 175 149 144 144
query92 59 54 57 54
query93 5251 568 604 568
query94 680 215 216 215
query95 1125 1082 1079 1079
query96 693 329 324 324
query97 6518 6362 6428 6362
query98 194 177 159 159
query99 2902 841 918 841
Total cold run time: 316275 ms
Total hot run time: 206004 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219511902
<details>
<summary>TPC-DS: <b>Total hot run time: 204307 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 50eb965b72dd6379918639419f8bc86330d5a624, data reload: false
query1 927 397 416 397
query2 6546 2747 2699 2699
query3 6925 219 204 204
query4 22093 18223 18669 18223
query5 20140 6644 6681 6644
query6 1125 233 249 233
query7 6650 325 332 325
query8 462 411 448 411
query9 3999 2684 2667 2667
query10 1272 313 306 306
query11 12207 10732 10672 10672
query12 129 82 74 74
query13 5608 701 707 701
query14 19425 13369 13649 13369
query15 365 233 252 233
query16 6427 288 265 265
query17 1481 1394 893 893
query18 2258 414 415 414
query19 217 153 157 153
query20 82 81 84 81
query21 196 98 99 98
query22 5240 5010 5078 5010
query23 32653 31991 32028 31991
query24 6790 6495 6480 6480
query25 526 438 430 430
query26 494 164 163 163
query27 1601 295 306 295
query28 6225 2368 2303 2303
query29 2911 2881 2841 2841
query30 245 166 173 166
query31 914 735 756 735
query32 69 65 61 61
query33 402 269 259 259
query34 847 479 470 470
query35 1126 929 925 925
query36 1336 1269 1239 1239
query37 89 61 62 61
query38 3120 2915 2920 2915
query39 1394 1336 1318 1318
query40 201 88 94 88
query41 55 43 45 43
query42 81 84 83 83
query43 763 759 694 694
query44 1125 739 736 736
query45 250 238 237 237
query46 1212 1000 990 990
query47 2003 1760 1718 1718
query48 1021 735 719 719
query49 619 378 370 370
query50 870 576 557 557
query51 4719 4651 4680 4651
query52 97 83 79 79
query53 453 324 324 324
query54 2662 2489 2446 2446
query55 88 77 90 77
query56 253 217 217 217
query57 1235 1133 1126 1126
query58 230 207 211 207
query59 4133 4045 3855 3855
query60 235 204 203 203
query61 100 94 95 94
query62 834 460 508 460
query63 486 347 347 347
query64 2493 1531 1515 1515
query65 3610 3584 3536 3536
query66 780 388 389 388
query67 15721 14878 15953 14878
query68 9090 648 648 648
query69 576 333 351 333
query70 1929 1408 1436 1408
query71 418 307 322 307
query72 6543 3509 3482 3482
query73 728 327 335 327
query74 6231 5882 5985 5882
query75 5503 3734 3707 3707
query76 5619 1190 1213 1190
query77 946 266 263 263
query78 12659 11877 11629 11629
query79 7556 664 636 636
query80 1013 411 408 408
query81 493 250 238 238
query82 1682 104 104 104
query83 174 133 131 131
query84 261 72 73 72
query85 876 332 336 332
query86 340 301 295 295
query87 3260 3046 3010 3010
query88 4634 2336 2333 2333
query89 354 282 266 266
query90 1935 218 216 216
query91 182 154 153 153
query92 62 60 56 56
query93 5472 596 588 588
query94 676 209 220 209
query95 1110 1059 1067 1059
query96 644 335 338 335
query97 6522 6273 6361 6273
query98 200 179 164 164
query99 2851 891 896 891
Total cold run time: 321384 ms
Total hot run time: 204307 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219499505
<details>
<summary>TPC-H: <b>Total hot run time: 50243 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 50eb965b72dd6379918639419f8bc86330d5a624, data reload: false
------ Round 1 ----------------------------------
q1 17765 4454 4414 4414
q2 2092 158 145 145
q3 10303 1911 1911 1911
q4 10228 1280 1313 1280
q5 8454 4321 3919 3919
q6 235 126 127 126
q7 2059 1635 1636 1635
q8 9420 2756 2736 2736
q9 10693 10352 10326 10326
q10 8636 3559 3514 3514
q11 423 254 251 251
q12 479 321 313 313
q13 18359 3946 4020 3946
q14 347 331 316 316
q15 505 467 458 458
q16 678 578 569 569
q17 1137 965 982 965
q18 7281 6908 6958 6908
q19 1808 1700 1664 1664
q20 524 315 287 287
q21 4512 4109 4155 4109
q22 527 455 451 451
Total cold run time: 116465 ms
Total hot run time: 50243 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4323 4339 4337 4337
q2 325 230 221 221
q3 4213 4178 4114 4114
q4 2754 2728 2746 2728
q5 7230 7139 7113 7113
q6 236 122 121 121
q7 3249 2854 2892 2854
q8 4377 4454 4547 4454
q9 16850 16789 16746 16746
q10 4286 4267 4276 4267
q11 773 675 664 664
q12 1027 885 889 885
q13 6893 3757 3753 3753
q14 461 424 422 422
q15 498 457 458 457
q16 755 709 685 685
q17 3878 3881 3844 3844
q18 8763 8686 8784 8686
q19 1751 1728 1649 1649
q20 2393 2163 2123 2123
q21 8530 8613 8581 8581
q22 1053 1051 1002 1002
Total cold run time: 84618 ms
Total hot run time: 79706 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219519680
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 50eb965b72dd6379918639419f8bc86330d5a624 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.1 seconds inserted 10000000 Rows, about 473K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219679617
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219729588
<details>
<summary>TPC-H: <b>Total hot run time: 49582 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e39cf9f4ac4efc17d111408afda61a5fd461bb3a, data reload: false
------ Round 1 ----------------------------------
q1 17990 4419 4329 4329
q2 2060 152 144 144
q3 10274 1856 1893 1856
q4 10357 1210 1345 1210
q5 8861 3852 3921 3852
q6 234 125 125 125
q7 2034 1578 1616 1578
q8 9322 2719 2690 2690
q9 10644 10369 10120 10120
q10 8637 3523 3522 3522
q11 414 248 244 244
q12 473 308 291 291
q13 18373 3988 4034 3988
q14 370 331 329 329
q15 505 462 467 462
q16 679 577 573 573
q17 1129 979 955 955
q18 7157 6892 6891 6891
q19 1771 1691 1616 1616
q20 515 314 313 313
q21 4441 4126 4056 4056
q22 535 438 449 438
Total cold run time: 116775 ms
Total hot run time: 49582 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4288 4286 4309 4286
q2 316 230 229 229
q3 4172 4141 4090 4090
q4 2728 2745 2748 2745
q5 7120 7080 7083 7080
q6 235 122 117 117
q7 3230 2828 2860 2828
q8 4356 4488 4475 4475
q9 16908 16754 16830 16754
q10 4228 4255 4281 4255
q11 774 699 686 686
q12 1018 844 848 844
q13 6437 3737 3709 3709
q14 461 419 422 419
q15 508 459 454 454
q16 736 681 694 681
q17 3853 3853 3828 3828
q18 8832 8692 8749 8692
q19 1735 1714 1686 1686
q20 2411 2146 2124 2124
q21 8664 8465 8467 8465
q22 1051 962 992 962
Total cold run time: 84061 ms
Total hot run time: 79409 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219802573
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 8a9934f3ac734f2b06b4f108f294b87c817a0ef8 with default session variables
Stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.2 seconds inserted 10000000 Rows, about 471K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2219786800
<details>
<summary>TPC-DS: <b>Total hot run time: 203419 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8a9934f3ac734f2b06b4f108f294b87c817a0ef8, data reload: false
query1 913 425 379 379
query2 5961 2832 2658 2658
query3 6578 208 202 202
query4 19937 18137 17937 17937
query5 18109 6502 6531 6502
query6 282 219 227 219
query7 4163 302 307 302
query8 462 472 386 386
query9 3084 2660 2592 2592
query10 404 304 285 285
query11 11293 10759 10803 10759
query12 121 82 73 73
query13 5634 713 682 682
query14 17357 13485 13924 13485
query15 360 245 240 240
query16 6456 282 261 261
query17 1682 1388 882 882
query18 2326 411 407 407
query19 204 147 153 147
query20 79 81 79 79
query21 190 94 98 94
query22 5284 5083 5071 5071
query23 32616 31895 31927 31895
query24 6973 6576 6559 6559
query25 513 425 425 425
query26 529 161 158 158
query27 1861 294 289 289
query28 6229 2357 2310 2310
query29 3044 2615 2656 2615
query30 246 169 164 164
query31 896 742 735 735
query32 70 65 60 60
query33 401 269 265 265
query34 842 471 466 466
query35 1115 939 941 939
query36 1276 1250 1118 1118
query37 94 63 61 61
query38 3095 2982 2959 2959
query39 1374 1340 1324 1324
query40 212 95 96 95
query41 50 43 44 43
query42 88 84 80 80
query43 784 731 731 731
query44 1123 728 717 717
query45 251 235 233 233
query46 1226 967 963 963
query47 1918 1760 1679 1679
query48 1005 707 711 707
query49 626 374 373 373
query50 871 601 618 601
query51 4800 4657 4656 4656
query52 90 80 79 79
query53 450 322 313 313
query54 2640 2454 2506 2454
query55 90 95 78 78
query56 227 222 210 210
query57 1216 1056 1257 1056
query58 215 195 206 195
query59 4106 4503 3902 3902
query60 215 221 217 217
query61 96 93 95 93
query62 797 505 483 483
query63 491 331 333 331
query64 2438 1496 1465 1465
query65 3638 3530 3555 3530
query66 762 375 374 374
query67 18174 15311 15563 15311
query68 9956 642 637 637
query69 581 331 337 331
query70 1692 1408 1359 1359
query71 399 313 306 306
query72 6486 3477 3418 3418
query73 734 333 322 322
query74 6317 5841 5816 5816
query75 5315 3705 3647 3647
query76 6124 1060 1114 1060
query77 1024 254 266 254
query78 12719 11718 13503 11718
query79 12668 653 623 623
query80 1162 391 396 391
query81 488 233 236 233
query82 900 95 95 95
query83 166 137 129 129
query84 260 70 71 70
query85 882 328 331 328
query86 353 338 293 293
query87 3245 3065 3024 3024
query88 5506 2332 2345 2332
query89 417 284 298 284
query90 1974 214 208 208
query91 178 139 151 139
query92 61 50 53 50
query93 5065 550 551 550
query94 750 205 206 205
query95 1088 1067 1035 1035
query96 645 330 325 325
query97 6441 6409 6289 6289
query98 192 181 170 170
query99 2744 825 885 825
Total cold run time: 317184 ms
Total hot run time: 203419 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2217494865
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #37415:
URL: https://github.com/apache/doris/pull/37415
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](txn_manager) Add ingested rowsets to unused rowsets when removing txn [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37415:
URL: https://github.com/apache/doris/pull/37415#issuecomment-2212927884
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit f0b38d19132e14d28106e77bc0d182f0f1d95d11 with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.1 seconds inserted 10000000 Rows, about 473K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org