You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Yukang-Lian (via GitHub)" <gi...@apache.org> on 2023/07/19 15:22:16 UTC
[GitHub] [doris] Yukang-Lian opened a new pull request, #22010: [Feature](Compaction) Support do full compaction by table id
Yukang-Lian opened a new pull request, #22010:
URL: https://github.com/apache/doris/pull/22010
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1646486828
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.28 seconds
stream load tsv: 505 seconds loaded 74807831229 Bytes, about 141 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.3 seconds inserted 10000000 Rows, about 341K ops/s
storage size: 17167255699 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1680595587
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1681915806
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1642306517
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] zhannngchen commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1681781228
pls also update the document `compaction-run.md`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] zhannngchen commented on a diff in pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on code in PR #22010:
URL: https://github.com/apache/doris/pull/22010#discussion_r1298057669
##########
docs/zh-CN/docs/admin-manual/http-actions/be/compaction-run.md:
##########
@@ -41,8 +42,11 @@ under the License.
* `tablet_id`
- tablet的id
+* `table_id`
+ - table的id。注意，table_id=xxx只有在compact_type=full时指定才会生效，并且tablet_id和table_id只能指定一个，不能够同时指定，指定table_id后会自动对此table下所有tablet执行full_compaction。
+
* `compact_type`
- - 取值为`base`或`cumulative`
+ - 取值为`base`或`cumulative`或`full`。full_compaction的使用场景请参考xxx。
Review Comment:
change to the correct link
##########
docs/zh-CN/docs/admin-manual/data-admin/data-recovery.md:
##########
@@ -0,0 +1,54 @@
+---
+{
+ "title": "数据恢复",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 数据恢复
+
+对于Merge on Write表，系统可能会在计算delete bitmap时出现错误，导致重复主键结果出现，此时可以利用full compaction功能进行数据的修复。本功能对于非Merge on Write表无效。
Review Comment:
Merge on Write表->Unique Key Merge on Write表
##########
docs/zh-CN/docs/admin-manual/data-admin/data-recovery.md:
##########
@@ -0,0 +1,54 @@
+---
+{
+ "title": "数据恢复",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 数据恢复
+
+对于Merge on Write表，系统可能会在计算delete bitmap时出现错误，导致重复主键结果出现，此时可以利用full compaction功能进行数据的修复。本功能对于非Merge on Write表无效。
Review Comment:
"系统可能会在计算delete bitmap时出现错误，导致重复主键结果出现"->"在某些Doris的版本中存在bug，可能会导致系统在计算delete bitmap时出现错误，导致出现重复主键“
##########
docs/zh-CN/docs/admin-manual/data-admin/data-recovery.md:
##########
@@ -0,0 +1,54 @@
+---
+{
+ "title": "数据恢复",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 数据恢复
+
+对于Merge on Write表，系统可能会在计算delete bitmap时出现错误，导致重复主键结果出现，此时可以利用full compaction功能进行数据的修复。本功能对于非Merge on Write表无效。
Review Comment:
"系统可能会在计算delete bitmap时出现错误，导致重复主键结果出现"->"在某些Doris的版本中存在bug，可能会导致系统在计算delete bitmap时出现错误，导致出现重复主键“
##########
docs/zh-CN/docs/admin-manual/data-admin/data-recovery.md:
##########
@@ -0,0 +1,54 @@
+---
+{
+ "title": "数据恢复",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# 数据恢复
+
+对于Merge on Write表，系统可能会在计算delete bitmap时出现错误，导致重复主键结果出现，此时可以利用full compaction功能进行数据的修复。本功能对于非Merge on Write表无效。
Review Comment:
Merge on Write表->Unique Key Merge on Write表
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1645257778
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] zhannngchen commented on a diff in pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on code in PR #22010:
URL: https://github.com/apache/doris/pull/22010#discussion_r1295777599
##########
be/src/http/action/compaction_action.cpp:
##########
@@ -95,44 +110,56 @@ Status CompactionAction::_handle_run_compaction(HttpRequest* req, std::string* j
return Status::NotSupported("The compaction type '{}' is not supported", compaction_type);
}
- // 2. fetch the tablet by tablet_id
- TabletSharedPtr tablet = StorageEngine::instance()->tablet_manager()->get_tablet(tablet_id);
- if (tablet == nullptr) {
- return Status::NotFound("Tablet not found. tablet_id={}", tablet_id);
- }
+ if (tablet_id == 0 && table_id != 0) {
Review Comment:
Should report error if user nether provide tablet id nor table id
##########
be/src/http/action/compaction_action.cpp:
##########
@@ -95,44 +110,56 @@ Status CompactionAction::_handle_run_compaction(HttpRequest* req, std::string* j
return Status::NotSupported("The compaction type '{}' is not supported", compaction_type);
}
- // 2. fetch the tablet by tablet_id
- TabletSharedPtr tablet = StorageEngine::instance()->tablet_manager()->get_tablet(tablet_id);
- if (tablet == nullptr) {
- return Status::NotFound("Tablet not found. tablet_id={}", tablet_id);
- }
+ if (tablet_id == 0 && table_id != 0) {
+ std::vector<TabletSharedPtr> tablet_vec =
+ StorageEngine::instance()->tablet_manager()->get_all_tablet(
+ [table_id](Tablet* tablet) -> bool {
+ return tablet->get_table_id() == table_id;
+ });
+ for (const auto& tablet : tablet_vec) {
+ StorageEngine::instance()->submit_compaction_task(
+ tablet, CompactionType::FULL_COMPACTION, false);
+ }
+ } else {
+ // 2. fetch the tablet by tablet_id
+ TabletSharedPtr tablet = StorageEngine::instance()->tablet_manager()->get_tablet(tablet_id);
+ if (tablet == nullptr) {
+ return Status::NotFound("Tablet not found. tablet_id={}", tablet_id);
+ }
- // 3. execute compaction task
- std::packaged_task<Status()> task([this, tablet, compaction_type]() {
- return _execute_compaction_callback(tablet, compaction_type);
- });
- std::future<Status> future_obj = task.get_future();
- std::thread(std::move(task)).detach();
+ // 3. execute compaction task
+ std::packaged_task<Status()> task([this, tablet, compaction_type]() {
+ return _execute_compaction_callback(tablet, compaction_type);
+ });
+ std::future<Status> future_obj = task.get_future();
+ std::thread(std::move(task)).detach();
- // 4. wait for result for 2 seconds by async
- std::future_status status = future_obj.wait_for(std::chrono::seconds(2));
- if (status == std::future_status::ready) {
- // fetch execute result
- Status olap_status = future_obj.get();
- if (!olap_status.ok()) {
- return olap_status;
+ // 4. wait for result for 2 seconds by async
+ std::future_status status = future_obj.wait_for(std::chrono::seconds(2));
+ if (status == std::future_status::ready) {
+ // fetch execute result
+ Status olap_status = future_obj.get();
+ if (!olap_status.ok()) {
+ return olap_status;
+ }
+ } else {
+ LOG(INFO) << "Manual compaction task is timeout for waiting "
+ << (status == std::future_status::timeout);
}
- } else {
- LOG(INFO) << "Manual compaction task is timeout for waiting "
- << (status == std::future_status::timeout);
}
-
LOG(INFO) << "Manual compaction task is successfully triggered";
*json_result =
- "{\"status\": \"Success\", \"msg\": \"compaction task is successfully triggered.\"}";
-
+ "{\"status\": \"Success\", \"msg\": \"compaction task is successfully "
Review Comment:
print table id or tablet id here
##########
be/src/http/action/compaction_action.cpp:
##########
@@ -95,44 +110,56 @@ Status CompactionAction::_handle_run_compaction(HttpRequest* req, std::string* j
return Status::NotSupported("The compaction type '{}' is not supported", compaction_type);
}
- // 2. fetch the tablet by tablet_id
- TabletSharedPtr tablet = StorageEngine::instance()->tablet_manager()->get_tablet(tablet_id);
- if (tablet == nullptr) {
- return Status::NotFound("Tablet not found. tablet_id={}", tablet_id);
- }
+ if (tablet_id == 0 && table_id != 0) {
Review Comment:
If user provide both of them, should report error as well
##########
be/src/olap/tablet.cpp:
##########
@@ -1858,14 +1878,29 @@ void Tablet::execute_compaction(CompactionType compaction_type) {
return;
}
set_last_base_compaction_failure_time(0);
+ } else {
+ // TODO: full compaction code
Review Comment:
outdated comment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1680590373
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Yukang-Lian commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1646447306
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1645156819
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Yukang-Lian commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1683232494
run clickbench
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Yukang-Lian commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1681755671
run clickbench
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] zhannngchen merged pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen merged PR #22010:
URL: https://github.com/apache/doris/pull/22010
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1681915481
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Yukang-Lian commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1680583063
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Yukang-Lian commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1681906096
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1683514963
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.69 seconds
stream load tsv: 535 seconds loaded 74807831229 Bytes, about 133 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.5 seconds inserted 10000000 Rows, about 338K ops/s
storage size: 17162186357 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1680590294
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Yukang-Lian commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "Yukang-Lian (via GitHub)" <gi...@apache.org>.
Yukang-Lian commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1683434029
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1645091697
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1645391176
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1644897455
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1646448491
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1683439837
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1683856523
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22010: [Feature](Compaction) Support do full compaction by table id
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22010:
URL: https://github.com/apache/doris/pull/22010#issuecomment-1680595683
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org