You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "platoneko (via GitHub)" <gi...@apache.org> on 2024/03/06 07:59:25 UTC
[PR] [enhance](io) Set segment file size to rowset meta [doris]
platoneko opened a new pull request, #31853:
URL: https://github.com/apache/doris/pull/31853
## Proposed changes
Set segment file size to rowset meta, to avoid the IO of fetching file meta info
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1996327162
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "platoneko (via GitHub)" <gi...@apache.org>.
platoneko commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1980767003
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "platoneko (via GitHub)" <gi...@apache.org>.
platoneko commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1993801944
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "platoneko (via GitHub)" <gi...@apache.org>.
platoneko commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1985118860
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1993905972
TeamCity be ut coverage result:
Function Coverage: 35.00% (8573/24492)
Line Coverage: 26.75% (69449/259620)
Region Coverage: 26.02% (36068/138605)
Branch Coverage: 22.98% (18418/80154)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c6a80d63b84a5f923fde7987aafcfa5c3248b993_c6a80d63b84a5f923fde7987aafcfa5c3248b993/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1980284542
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "platoneko (via GitHub)" <gi...@apache.org>.
platoneko commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1985097193
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1993896254
<details>
<summary>TPC-H: <b>Total hot run time: 36800 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c6a80d63b84a5f923fde7987aafcfa5c3248b993, data reload: false
------ Round 1 ----------------------------------
q1 17625 4955 4167 4167
q2 2020 149 148 148
q3 10581 965 901 901
q4 4649 706 679 679
q5 7604 2744 2862 2744
q6 182 123 129 123
q7 1237 807 802 802
q8 9501 2116 2055 2055
q9 7363 6488 6495 6488
q10 8249 2511 2513 2511
q11 426 231 217 217
q12 795 303 286 286
q13 17969 3030 3038 3030
q14 285 251 252 251
q15 501 454 447 447
q16 498 393 392 392
q17 966 566 623 566
q18 6692 5977 5817 5817
q19 1573 1389 1374 1374
q20 539 300 282 282
q21 7376 3339 3225 3225
q22 822 295 301 295
Total cold run time: 107453 ms
Total hot run time: 36800 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4106 4108 4129 4108
q2 324 223 226 223
q3 3014 2854 2853 2853
q4 1860 1603 1543 1543
q5 5231 5332 5248 5248
q6 198 120 118 118
q7 2292 1817 1851 1817
q8 3170 3306 3345 3306
q9 8570 8525 8599 8525
q10 6116 3690 3699 3690
q11 536 432 456 432
q12 718 544 566 544
q13 13897 2875 2879 2875
q14 276 260 259 259
q15 488 454 441 441
q16 455 419 422 419
q17 1748 1503 1476 1476
q18 7709 7299 7131 7131
q19 6924 1611 1548 1548
q20 1913 1690 1699 1690
q21 4725 4751 4770 4751
q22 484 446 443 443
Total cold run time: 74754 ms
Total hot run time: 53440 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #31853:
URL: https://github.com/apache/doris/pull/31853#discussion_r1514051381
##########
be/src/olap/rowset/beta_rowset_writer.cpp:
##########
@@ -664,6 +739,13 @@ Status BaseBetaRowsetWriter::_build_rowset_meta(RowsetMeta* rowset_meta, bool ch
// TODO write zonemap to meta
rowset_meta->set_empty((num_rows_written + _num_rows_written) == 0);
rowset_meta->set_creation_time(time(nullptr));
+
+ if (auto seg_file_size = _seg_files.segments_file_size(); !seg_file_size.has_value()) {
+ LOG(ERROR) << seg_file_size.error();
Review Comment:
Should we return an error status?
##########
be/src/olap/rowset/rowset_meta.h:
##########
@@ -314,6 +314,12 @@ class RowsetMeta {
void set_txn_expiration(int64_t expiration) { _rowset_meta_pb.set_txn_expiration(expiration); }
+ // `seg_file_size` MUST ordered by segment id
+ void set_segments_file_size(const std::vector<size_t>& seg_file_size);
Review Comment:
how to ensure
> `seg_file_size` MUST ordered by segment id
should we check the order or sort it in `set_segments_file_size`
##########
be/src/olap/rowset/beta_rowset_writer.cpp:
##########
@@ -96,12 +97,92 @@ void build_rowset_meta_with_spec_field(RowsetMeta& rowset_meta,
} // namespace
+SegmentFileCollection::~SegmentFileCollection() = default;
+
+Status SegmentFileCollection::add(int seg_id, io::FileWriterPtr&& writer) {
+ std::lock_guard lock(_lock);
+ if (_closed) [[unlikely]] {
+ DCHECK(false) << writer->path();
+ return Status::InternalError("add to closed SegmentFileCollection");
+ }
+
+ _file_writers.emplace_back(seg_id, std::move(writer));
+ return Status::OK();
+}
+
+Status SegmentFileCollection::close() {
+ {
+ std::lock_guard lock(_lock);
+ if (_closed) [[unlikely]] {
+ DCHECK(false);
+ return Status::InternalError("double close SegmentFileCollection");
+ }
+ _closed = true;
+ }
+
+ for (auto&& [_, writer] : _file_writers) {
+ RETURN_IF_ERROR(writer->close());
+ }
+
+ return Status::OK();
+}
+
+Result<std::vector<size_t>> SegmentFileCollection::segments_file_size() {
+ std::lock_guard lock(_lock);
+ if (!_closed) [[unlikely]] {
+ DCHECK(false);
+ return ResultError(Status::InternalError("get segments file size without closed"));
+ }
+
+ Status st;
+ std::vector<size_t> seg_file_size(_file_writers.size(), 0);
+ bool succ = std::all_of(_file_writers.begin(), _file_writers.end(), [&](auto&& it) {
+ auto&& [seg_id, writer] = it;
+
+ if (seg_id > seg_file_size.size()) [[unlikely]] {
Review Comment:
it seems to be `seg_id >= seg_file_size.size()` ?
##########
be/src/olap/rowset/vertical_beta_rowset_writer.h:
##########
@@ -17,28 +17,28 @@
#pragma once
-#include <stddef.h>
-#include <stdint.h>
-
#include <memory>
+#include <type_traits>
#include <vector>
#include "common/status.h"
#include "olap/rowset/beta_rowset_writer.h"
#include "olap/rowset/segment_v2/segment_writer.h"
-#include "olap/rowset/vertical_beta_rowset_writer_helper.h"
namespace doris {
namespace vectorized {
class Block;
} // namespace vectorized
// for vertical compaction
-// TODO(plat1ko): Inherited from template type `T`, `T` is `BetaRowsetWriter` or `CloudBetaRowsetWriter`
-class VerticalBetaRowsetWriter final : public BetaRowsetWriter {
+template <class T>
+ requires std::is_base_of_v<BaseBetaRowsetWriter, T>
+class VerticalBetaRowsetWriter final : public T {
Review Comment:
does gcc compile?
##########
be/src/olap/rowset/beta_rowset_writer.cpp:
##########
@@ -96,12 +97,92 @@ void build_rowset_meta_with_spec_field(RowsetMeta& rowset_meta,
} // namespace
+SegmentFileCollection::~SegmentFileCollection() = default;
+
+Status SegmentFileCollection::add(int seg_id, io::FileWriterPtr&& writer) {
+ std::lock_guard lock(_lock);
+ if (_closed) [[unlikely]] {
+ DCHECK(false) << writer->path();
+ return Status::InternalError("add to closed SegmentFileCollection");
+ }
+
+ _file_writers.emplace_back(seg_id, std::move(writer));
+ return Status::OK();
+}
+
+Status SegmentFileCollection::close() {
+ {
+ std::lock_guard lock(_lock);
+ if (_closed) [[unlikely]] {
+ DCHECK(false);
+ return Status::InternalError("double close SegmentFileCollection");
+ }
+ _closed = true;
+ }
+
+ for (auto&& [_, writer] : _file_writers) {
+ RETURN_IF_ERROR(writer->close());
+ }
+
+ return Status::OK();
+}
+
+Result<std::vector<size_t>> SegmentFileCollection::segments_file_size() {
+ std::lock_guard lock(_lock);
+ if (!_closed) [[unlikely]] {
+ DCHECK(false);
+ return ResultError(Status::InternalError("get segments file size without closed"));
+ }
+
+ Status st;
+ std::vector<size_t> seg_file_size(_file_writers.size(), 0);
+ bool succ = std::all_of(_file_writers.begin(), _file_writers.end(), [&](auto&& it) {
+ auto&& [seg_id, writer] = it;
+
+ if (seg_id > seg_file_size.size()) [[unlikely]] {
+ auto err_msg = fmt::format("invalid seg_id={} num_file_writers={} path={}", seg_id,
+ seg_file_size.size(), writer->path().native());
+ DCHECK(false) << err_msg;
+ st = Status::InternalError(err_msg);
+ return false;
+ }
+
+ auto& fsize = seg_file_size[seg_id];
+ if (fsize != 0) {
+ // File size should not been set
+ auto err_msg =
+ fmt::format("duplicate seg_id={} path={}", seg_id, writer->path().native());
+ DCHECK(false) << err_msg;
+ st = Status::InternalError(err_msg);
+ return false;
+ }
+
+ fsize = writer->bytes_appended();
+ if (fsize <= 0) {
+ auto err_msg =
+ fmt::format("invalid segment fsize={} path={}", fsize, writer->path().native());
+ DCHECK(false) << err_msg;
+ st = Status::InternalError(err_msg);
+ return false;
+ }
+
+ return true;
+ });
+
+ if (succ) {
Review Comment:
try ternary expr. to reduce lines of code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "platoneko (via GitHub)" <gi...@apache.org>.
platoneko commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1980285293
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1980475093
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "platoneko (via GitHub)" <gi...@apache.org>.
platoneko commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1993768869
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1993925924
<details>
<summary>TPC-DS: <b>Total hot run time: 187572 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c6a80d63b84a5f923fde7987aafcfa5c3248b993, data reload: false
query1 919 360 348 348
query2 7409 1950 1957 1950
query3 6707 214 207 207
query4 31145 20447 20675 20447
query5 4311 401 477 401
query6 267 173 170 170
query7 4634 311 290 290
query8 230 166 167 166
query9 9878 2305 2272 2272
query10 446 232 242 232
query11 17086 13996 14379 13996
query12 135 87 83 83
query13 1617 420 419 419
query14 11603 10843 10719 10719
query15 256 210 188 188
query16 7932 271 266 266
query17 1938 569 538 538
query18 2022 288 283 283
query19 227 160 154 154
query20 93 83 85 83
query21 204 129 123 123
query22 4628 4481 4495 4481
query23 31542 31081 31287 31081
query24 10756 2855 2850 2850
query25 640 377 393 377
query26 1267 160 163 160
query27 2627 348 355 348
query28 7786 1930 1889 1889
query29 922 629 625 625
query30 308 149 150 149
query31 967 717 726 717
query32 96 60 56 56
query33 768 265 249 249
query34 1124 479 478 478
query35 830 637 611 611
query36 975 847 863 847
query37 125 74 78 74
query38 3296 3229 3168 3168
query39 1425 1392 1412 1392
query40 205 110 114 110
query41 51 47 51 47
query42 101 97 96 96
query43 479 441 426 426
query44 1281 702 725 702
query45 271 249 260 249
query46 1079 713 699 699
query47 1683 1619 1619 1619
query48 441 367 357 357
query49 1141 330 337 330
query50 782 383 398 383
query51 6843 6760 6752 6752
query52 104 92 92 92
query53 346 276 278 276
query54 308 254 248 248
query55 89 88 77 77
query56 242 253 239 239
query57 1062 1000 1032 1000
query58 246 202 223 202
query59 2871 2575 2537 2537
query60 267 242 252 242
query61 111 111 147 111
query62 621 387 408 387
query63 298 275 270 270
query64 5316 3907 3863 3863
query65 3087 3014 3031 3014
query66 838 328 347 328
query67 15120 14435 14739 14435
query68 7988 535 541 535
query69 645 388 385 385
query70 1237 1183 1194 1183
query71 539 283 278 278
query72 6763 2634 2509 2509
query73 738 315 320 315
query74 7317 6586 6610 6586
query75 9671 8185 8089 8089
query76 5002 930 1002 930
query77 679 272 261 261
query78 10600 9646 9553 9553
query79 11238 539 536 536
query80 1918 388 390 388
query81 530 212 207 207
query82 876 197 200 197
query83 214 143 143 143
query84 283 79 77 77
query85 1495 318 315 315
query86 480 311 316 311
query87 3479 3276 3249 3249
query88 5344 2363 2343 2343
query89 515 359 357 357
query90 1982 188 183 183
query91 160 130 140 130
query92 64 46 48 46
query93 7417 512 496 496
query94 1227 181 181 181
query95 432 327 327 327
query96 609 268 279 268
query97 4432 4262 4297 4262
query98 231 213 205 205
query99 1089 749 753 749
Total cold run time: 319618 ms
Total hot run time: 187572 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1993960938
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit c6a80d63b84a5f923fde7987aafcfa5c3248b993 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.9 seconds inserted 10000000 Rows, about 719K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #31853:
URL: https://github.com/apache/doris/pull/31853
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "platoneko (via GitHub)" <gi...@apache.org>.
platoneko commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1985128911
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1985216075
TeamCity be ut coverage result:
Function Coverage: 35.37% (8553/24183)
Line Coverage: 27.14% (69497/256104)
Region Coverage: 26.36% (36079/136849)
Branch Coverage: 23.21% (18438/79454)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1161cfbdfc840b3493efe1c84dfa6ce678ea4295_1161cfbdfc840b3493efe1c84dfa6ce678ea4295/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1980902317
TeamCity be ut coverage result:
Function Coverage: 35.40% (8557/24169)
Line Coverage: 27.15% (69463/255855)
Region Coverage: 26.38% (36043/136648)
Branch Coverage: 23.21% (18416/79346)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f8bfa28a1e308b3b755128dc2480f28314226dd8_f8bfa28a1e308b3b755128dc2480f28314226dd8/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "platoneko (via GitHub)" <gi...@apache.org>.
platoneko commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1980290828
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhance](io) Set segment file size to rowset meta [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31853:
URL: https://github.com/apache/doris/pull/31853#issuecomment-1993955569
<details>
<summary>ClickBench: <b>Total hot run time: 30.12 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c6a80d63b84a5f923fde7987aafcfa5c3248b993, data reload: false
query1 0.04 0.03 0.03
query2 0.06 0.03 0.03
query3 0.23 0.06 0.06
query4 1.67 0.10 0.10
query5 0.53 0.50 0.48
query6 1.23 0.63 0.63
query7 0.02 0.01 0.01
query8 0.04 0.03 0.02
query9 0.56 0.50 0.48
query10 0.54 0.53 0.55
query11 0.11 0.09 0.09
query12 0.11 0.10 0.09
query13 0.56 0.57 0.56
query14 0.73 0.76 0.78
query15 0.79 0.78 0.77
query16 0.39 0.39 0.40
query17 0.98 1.00 0.98
query18 0.21 0.24 0.24
query19 1.78 1.69 1.67
query20 0.02 0.01 0.01
query21 15.40 0.55 0.56
query22 2.60 1.73 2.11
query23 17.10 1.05 0.74
query24 3.17 2.92 0.90
query25 0.21 0.12 0.03
query26 0.81 0.13 0.12
query27 0.04 0.03 0.02
query28 9.96 0.89 0.87
query29 12.49 3.29 3.16
query30 0.62 0.56 0.60
query31 2.81 0.34 0.34
query32 3.37 0.44 0.42
query33 2.84 2.77 2.81
query34 16.05 4.37 4.32
query35 4.34 4.41 4.35
query36 1.06 1.00 1.01
query37 0.07 0.06 0.06
query38 0.05 0.03 0.03
query39 0.03 0.02 0.02
query40 0.17 0.15 0.14
query41 0.07 0.02 0.02
query42 0.03 0.02 0.02
query43 0.02 0.03 0.02
Total cold run time: 103.91 s
Total hot run time: 30.12 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org