You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yujun777 (via GitHub)" <gi...@apache.org> on 2023/11/04 16:33:02 UTC
[PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
yujun777 opened a new pull request, #26428:
URL: https://github.com/apache/doris/pull/26428
Concurrent schema change and txn may cause dead lock. An example:
1. Txn T commit but not publish;
2. Run schema change or rollup on T's related partition, add alter replica R;
3. sc/rollup add a sched txn watermark M;
4. Restart fe;
5. After fe restart, T's loadedTblIndexes will clear loadedTblIndexes because it's not save to disk;
6. T will publish version to all tablet, including sc/rollup's new alter replica R;
7. Since R not contains txn data, so the T will fail. It will then always waitting for R's data;
8. sc/rollup wait for txn before M to finish, only after it will let R copy history data;
9. Since T's not finished, so sc/rollup will always wait, so R will nerver copy history data;
10. R and sc/rollup will wait each other forever, cause dead lock;
Fix:
For sc/rollup, it will ensure double write after the sched watermark M, so for finish transaction, when checking a alter replica:
1. if txn id is bigger than M, check it just like a normal replica;
2. otherwise skip check this replica, the BE will modify history data later.
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "deardeng (via GitHub)" <gi...@apache.org>.
deardeng commented on code in PR #26428:
URL: https://github.com/apache/doris/pull/26428#discussion_r1382944155
##########
regression-test/suites/schema_change/test_schema_change_concurrent_with_txn.groovy:
##########
@@ -0,0 +1,135 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+import org.apache.doris.regression.suite.ClusterOptions
+import org.apache.doris.regression.util.NodeType
+
+suite('test_schema_change_concurrent_with_txn') {
+ def options = new ClusterOptions()
+ options.enableDebugPoints()
+ options.feConfigs.add('publish_wait_time_second=-1')
+ docker(options) {
+ sql 'SET GLOBAL insert_visible_timeout_ms = 2000'
+
+ def result = sql 'SELECT DATABASE()'
+ def dbName = result[0][0]
+
+ sql 'CREATE TABLE tbl_1 (k1 INT, k2 INT) PROPERTIES ( "light_schema_change" = "false")'
+ sql 'INSERT INTO tbl_1 VALUES (1, 10)'
+ sql 'INSERT INTO tbl_1 VALUES (2, 20)'
+ order_qt_select_1_1 'SELECT * FROM tbl_1'
+
+ sql 'CREATE TABLE tbl_2 AS SELECT * FROM tbl_1'
+ order_qt_select_2_1 'SELECT * FROM tbl_1'
Review Comment:
tbl_2
##########
fe/fe-core/src/main/java/org/apache/doris/transaction/DatabaseTransactionMgr.java:
##########
@@ -1111,8 +1114,28 @@ public void finishTransaction(long transactionId) throws UserException {
LOG.info("finish transaction {} successfully, publish result: {}", transactionState, publishResult.name());
}
- private void checkReplicaContinuousVersionSucc(long tabletId, Replica replica, long version,
- PublishVersionTask backendPublishTask, Set<Long> errorReplicaIds, List<Replica> tabletSuccReplicas,
+ private boolean isCheckTxnIgnoreAlterReplica(long transactionId, OlapTable table) {
Review Comment:
function name publishSkipAlterReplica ?
##########
fe/fe-core/src/main/java/org/apache/doris/transaction/DatabaseTransactionMgr.java:
##########
@@ -1111,8 +1114,28 @@ public void finishTransaction(long transactionId) throws UserException {
LOG.info("finish transaction {} successfully, publish result: {}", transactionState, publishResult.name());
}
- private void checkReplicaContinuousVersionSucc(long tabletId, Replica replica, long version,
- PublishVersionTask backendPublishTask, Set<Long> errorReplicaIds, List<Replica> tabletSuccReplicas,
+ private boolean isCheckTxnIgnoreAlterReplica(long transactionId, OlapTable table) {
Review Comment:
function name publishSkipAlterReplica ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1793500570
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.09 seconds
stream load tsv: 556 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17161970115 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1803041240
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on code in PR #26428:
URL: https://github.com/apache/doris/pull/26428#discussion_r1387394834
##########
fe/fe-core/src/main/java/org/apache/doris/transaction/DatabaseTransactionMgr.java:
##########
@@ -1111,8 +1114,28 @@ public void finishTransaction(long transactionId) throws UserException {
LOG.info("finish transaction {} successfully, publish result: {}", transactionState, publishResult.name());
}
- private void checkReplicaContinuousVersionSucc(long tabletId, Replica replica, long version,
- PublishVersionTask backendPublishTask, Set<Long> errorReplicaIds, List<Replica> tabletSuccReplicas,
+ private boolean isCheckTxnIgnoreAlterReplica(long transactionId, OlapTable table) {
Review Comment:
updated
##########
be/src/olap/task/engine_publish_version_task.cpp:
##########
@@ -235,7 +236,19 @@ Status EnginePublishVersionTask::finish() {
} else {
// check if the version exist, if not exist, then set publish failed
if (_error_tablet_ids->find(tablet_id) == _error_tablet_ids->end()) {
- if (tablet->check_version_exist(version)) {
+ bool exist_version = tablet->check_version_exist(version);
+ bool is_converting = false;
+ if (!exist_version && tablet->tablet_state() == TabletState::TABLET_NOTREADY) {
+ bool is_mow = tablet->keys_type() == KeysType::UNIQUE_KEYS &&
Review Comment:
remove be code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1793622369
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1807177581
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.86 seconds
stream load tsv: 554 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162073708 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1803126239
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 42d877c6cd1eb95725ab9798902b8638bcd1791d, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5199 5170 5048 5048
q2 374 244 228 228
q3 2059 2038 2029 2029
q4 1466 1459 1430 1430
q5 4161 4154 4096 4096
q6 248 133 140 133
q7 2086 1598 1612 1598
q8 2739 2731 2729 2729
q9 10679 10344 10287 10287
q10 3488 3579 3577 3577
q11 372 249 259 249
q12 455 303 305 303
q13 4525 4088 4130 4088
q14 326 291 284 284
q15 630 559 577 559
q16 697 623 585 585
q17 1135 1079 1058 1058
q18 7848 7369 7453 7369
q19 1685 1692 1683 1683
q20 578 366 352 352
q21 4953 4573 4556 4556
q22 530 416 420 416
Total cold run time: 56233 ms
Total hot run time: 52657 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4960 4967 4933 4933
q2 349 247 231 231
q3 4021 4045 4038 4038
q4 2800 2796 2748 2748
q5 6504 6461 6436 6436
q6 238 126 130 126
q7 3111 2728 2702 2702
q8 4767 4713 4742 4713
q9 17692 17803 17602 17602
q10 4080 4181 4199 4181
q11 815 639 668 639
q12 984 802 812 802
q13 4319 3912 3887 3887
q14 388 343 361 343
q15 606 548 541 541
q16 769 665 684 665
q17 3968 3838 3984 3838
q18 9691 9226 9273 9226
q19 1898 1764 1763 1763
q20 2398 2060 2048 2048
q21 8889 8958 8869 8869
q22 918 870 880 870
Total cold run time: 84165 ms
Total hot run time: 81201 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1808179349
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1793496646
TeamCity be ut coverage result:
Function Coverage: 37.29% (8452/22666)
Line Coverage: 29.67% (68433/230643)
Region Coverage: 28.32% (35404/124994)
Branch Coverage: 25.07% (18075/72102)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b8fa48654ec65a65f8ed5fe04da4c2c1b32313a9_b8fa48654ec65a65f8ed5fe04da4c2c1b32313a9/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1793623774
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1807174828
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit d747c5e3feb862820dbe120a77abe71857e42192, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5170 5021 5174 5021
q2 369 204 198 198
q3 2078 2065 2055 2055
q4 1477 1436 1455 1436
q5 4115 4157 4083 4083
q6 253 130 132 130
q7 2056 1604 1608 1604
q8 2769 2744 2748 2744
q9 10480 10303 10267 10267
q10 10247 3592 3551 3551
q11 363 248 268 248
q12 467 293 289 289
q13 4528 4131 4108 4108
q14 315 283 295 283
q15 618 569 579 569
q16 696 625 606 606
q17 1134 1066 1080 1066
q18 7822 7469 7458 7458
q19 1715 1693 1690 1690
q20 589 372 366 366
q21 4941 4579 4593 4579
q22 549 441 438 438
Total cold run time: 62751 ms
Total hot run time: 52789 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5093 5023 5106 5023
q2 366 250 251 250
q3 4022 4025 3978 3978
q4 2793 2746 2720 2720
q5 6531 6432 6455 6432
q6 237 126 129 126
q7 3144 2742 2729 2729
q8 4783 4864 4807 4807
q9 17738 17724 17701 17701
q10 4078 4135 4173 4135
q11 764 657 664 657
q12 991 848 825 825
q13 4323 3894 3885 3885
q14 385 348 346 346
q15 660 595 569 569
q16 785 695 682 682
q17 3928 3834 3905 3834
q18 9393 9266 9140 9140
q19 1879 1756 1778 1756
q20 2391 2034 2037 2034
q21 8789 8715 8780 8715
q22 1008 875 867 867
Total cold run time: 84081 ms
Total hot run time: 81211 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #26428:
URL: https://github.com/apache/doris/pull/26428
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1793491782
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1803046096
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "deardeng (via GitHub)" <gi...@apache.org>.
deardeng commented on code in PR #26428:
URL: https://github.com/apache/doris/pull/26428#discussion_r1383006786
##########
be/src/olap/task/engine_publish_version_task.cpp:
##########
@@ -235,7 +236,19 @@ Status EnginePublishVersionTask::finish() {
} else {
// check if the version exist, if not exist, then set publish failed
if (_error_tablet_ids->find(tablet_id) == _error_tablet_ids->end()) {
- if (tablet->check_version_exist(version)) {
+ bool exist_version = tablet->check_version_exist(version);
+ bool is_converting = false;
+ if (!exist_version && tablet->tablet_state() == TabletState::TABLET_NOTREADY) {
+ bool is_mow = tablet->keys_type() == KeysType::UNIQUE_KEYS &&
Review Comment:
add some log?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1807105892
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1803122378
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.01 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162137592 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1793629989
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.51 seconds
stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17162153798 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1793628867
TeamCity be ut coverage result:
Function Coverage: 37.29% (8452/22666)
Line Coverage: 29.67% (68432/230643)
Region Coverage: 28.33% (35412/124994)
Branch Coverage: 25.06% (18072/72102)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2c44b7f5274e958448b3e92c5bb7b9e7bc681afd_2c44b7f5274e958448b3e92c5bb7b9e7bc681afd/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "luwei16 (via GitHub)" <gi...@apache.org>.
luwei16 commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1801243781
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1807107526
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1808179597
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1803039718
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1803075607
TeamCity be ut coverage result:
Function Coverage: 36.69% (8388/22864)
Line Coverage: 29.21% (68023/232915)
Region Coverage: 27.86% (35181/126256)
Branch Coverage: 24.68% (17986/72866)
Coverage Report: http://coverage.selectdb-in.cc/coverage/42d877c6cd1eb95725ab9798902b8638bcd1791d_42d877c6cd1eb95725ab9798902b8638bcd1791d/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1793490228
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](transaction) Fix concurrent schema change and txn cause dead lock [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #26428:
URL: https://github.com/apache/doris/pull/26428#issuecomment-1807385765
TeamCity be ut coverage result:
Function Coverage: 36.78% (8407/22855)
Line Coverage: 29.29% (68161/232730)
Region Coverage: 27.91% (35224/126202)
Branch Coverage: 24.73% (18015/72842)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d747c5e3feb862820dbe120a77abe71857e42192_d747c5e3feb862820dbe120a77abe71857e42192/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org