You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "xinyiZzz (via GitHub)" <gi...@apache.org> on 2023/12/07 16:29:20 UTC
[PR] [fix](memory) Warning in log when turning on THP [doris]
xinyiZzz opened a new pull request, #28149:
URL: https://github.com/apache/doris/pull/28149
## Proposed changes
1. LRU Cache of type `LRUCacheType::NUMBER` no longer records memory Bytes separately.
Only records entry number, because using `LRUCacheType::NUMBER` type is usually due to inability to accurately track memory.
2. LRU Cache of type `LRUCacheType::NUMBER` cache capacity means the number of entries, so the `charge` of insert is always `1` instead of `sizeof(CacheValue)`, which is easier to understand.
3. `TabletSchema` and `Schema` no longer track memory, only track columns count. because cannot accurately track memory size.
4. Segment::_meta_mem_usage Unknown value overflow, causes the value of SegmentMeta MemTracker is similar to `-2912341218700198079`. So, temporarily put it in experimental type tracker.
5. TabletMeta MemTracker changed to track TabletSchema columns count.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #28149:
URL: https://github.com/apache/doris/pull/28149#discussion_r1420185314
##########
be/src/olap/tablet_manager.cpp:
##########
@@ -236,10 +234,12 @@ Status TabletManager::_add_tablet_to_map_unlocked(TTabletId tablet_id,
tablet_map_t& tablet_map = _get_tablet_map(tablet_id);
tablet_map[tablet_id] = tablet;
_add_tablet_to_partition(tablet);
+ // TODO: memory size of TabletSchema cannot be accurately tracked.
// TODO: remove multiply 2 of tablet meta mem size
// Because table schema will copy in tablet, there will be double mem cost
// so here multiply 2
- _tablet_meta_mem_tracker->consume(tablet->tablet_meta()->mem_size() * 2);
+ // _tablet_meta_schema_columns_tracker->consume(tablet->tablet_meta()->mem_size() * 2);
+ _tablet_meta_schema_columns_tracker->consume(tablet->tablet_meta()->tablet_columns_num());
Review Comment:
这既然不是一个memtracker， 就不要用memtracker 来统计个数了，后面会翻车。
如果他只是一个counter，那么就用类似bvar或者doris metric 这种方式直接作为监控项来统计。
##########
be/src/olap/tablet_manager.cpp:
##########
@@ -236,10 +234,12 @@ Status TabletManager::_add_tablet_to_map_unlocked(TTabletId tablet_id,
tablet_map_t& tablet_map = _get_tablet_map(tablet_id);
tablet_map[tablet_id] = tablet;
_add_tablet_to_partition(tablet);
+ // TODO: memory size of TabletSchema cannot be accurately tracked.
// TODO: remove multiply 2 of tablet meta mem size
// Because table schema will copy in tablet, there will be double mem cost
// so here multiply 2
- _tablet_meta_mem_tracker->consume(tablet->tablet_meta()->mem_size() * 2);
+ // _tablet_meta_schema_columns_tracker->consume(tablet->tablet_meta()->mem_size() * 2);
+ _tablet_meta_schema_columns_tracker->consume(tablet->tablet_meta()->tablet_columns_num());
Review Comment:
这既然不是一个memtracker， 就不要用memtracker 来统计个数了，后面会翻车。
如果他只是一个counter，那么就用类似bvar或者doris metric 这种方式直接作为监控项来统计。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) Warning in log when turning on THP [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #28149:
URL: https://github.com/apache/doris/pull/28149#discussion_r1419260423
##########
be/src/olap/tablet_meta.cpp:
##########
@@ -679,7 +679,7 @@ void TabletMeta::to_meta_pb(TabletMetaPB* tablet_meta_pb) {
time_series_compaction_time_threshold_seconds());
}
-uint32_t TabletMeta::mem_size() const {
+int64_t TabletMeta::mem_size() const {
Review Comment:
warning: method 'mem_size' can be made static [readability-convert-member-functions-to-static]
```suggestion
int64_t TabletMeta::mem_size() {
```
be/src/olap/tablet_meta.h:136:
```diff
- int64_t mem_size() const;
+ static int64_t mem_size() ;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on code in PR #28149:
URL: https://github.com/apache/doris/pull/28149#discussion_r1423714318
##########
be/src/olap/tablet_meta.cpp:
##########
@@ -679,7 +679,7 @@ void TabletMeta::to_meta_pb(TabletMetaPB* tablet_meta_pb) {
time_series_compaction_time_threshold_seconds());
}
-uint32_t TabletMeta::mem_size() const {
+int64_t TabletMeta::mem_size() const {
Review Comment:
先记着TODO留着吧，假以时日，还是要把元数据内存统计准的
##########
be/src/olap/tablet_meta.cpp:
##########
@@ -679,7 +679,7 @@ void TabletMeta::to_meta_pb(TabletMetaPB* tablet_meta_pb) {
time_series_compaction_time_threshold_seconds());
}
-uint32_t TabletMeta::mem_size() const {
+int64_t TabletMeta::mem_size() const {
Review Comment:
先记着TODO留着吧，假以时日，还是要把元数据内存统计准的
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #28149:
URL: https://github.com/apache/doris/pull/28149#discussion_r1420186128
##########
be/src/olap/tablet_meta.cpp:
##########
@@ -679,7 +679,7 @@ void TabletMeta::to_meta_pb(TabletMetaPB* tablet_meta_pb) {
time_series_compaction_time_threshold_seconds());
}
-uint32_t TabletMeta::mem_size() const {
+int64_t TabletMeta::mem_size() const {
Review Comment:
如果没有用的话，直接把这个代码删了吧
##########
be/src/olap/tablet_meta.cpp:
##########
@@ -679,7 +679,7 @@ void TabletMeta::to_meta_pb(TabletMetaPB* tablet_meta_pb) {
time_series_compaction_time_threshold_seconds());
}
-uint32_t TabletMeta::mem_size() const {
+int64_t TabletMeta::mem_size() const {
Review Comment:
如果没有用的话，直接把这个代码删了吧
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1846533497
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit c7e7f258e38f889c7231768cedc6510cb8f87469, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4676 4427 4491 4427
q2 366 141 158 141
q3 1457 1214 1187 1187
q4 1106 899 900 899
q5 3132 3102 3136 3102
q6 250 132 130 130
q7 967 494 484 484
q8 2194 2228 2166 2166
q9 6655 7005 6621 6621
q10 3240 3249 3248 3248
q11 329 202 209 202
q12 344 204 207 204
q13 4831 3856 3802 3802
q14 245 210 221 210
q15 561 528 527 527
q16 455 384 394 384
q17 1003 570 516 516
q18 7427 7318 7192 7192
q19 1512 1383 1394 1383
q20 1973 322 305 305
q21 3110 2609 2644 2609
q22 352 279 289 279
Total cold run time: 46185 ms
Total hot run time: 40018 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4402 4413 4397 4397
q2 269 161 167 161
q3 3537 3529 3533 3529
q4 2379 2368 2361 2361
q5 5731 5754 5735 5735
q6 241 118 123 118
q7 2397 1844 1856 1844
q8 3512 3503 3527 3503
q9 9070 9047 9031 9031
q10 3913 3993 3987 3987
q11 510 396 367 367
q12 765 578 602 578
q13 4296 3587 3589 3587
q14 278 260 257 257
q15 580 520 519 519
q16 502 451 458 451
q17 1865 1848 1876 1848
q18 8604 8343 8634 8343
q19 1731 1733 1795 1733
q20 2283 1955 1932 1932
q21 6491 6183 6155 6155
q22 496 419 418 418
Total cold run time: 63852 ms
Total hot run time: 60854 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "Lchangliang (via GitHub)" <gi...@apache.org>.
Lchangliang commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1853370054
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1846534776
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.74 seconds
stream load tsv: 590 seconds loaded 74807831229 Bytes, about 120 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17220622567 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on code in PR #28149:
URL: https://github.com/apache/doris/pull/28149#discussion_r1423843072
##########
be/src/olap/tablet_manager.cpp:
##########
@@ -236,10 +234,12 @@ Status TabletManager::_add_tablet_to_map_unlocked(TTabletId tablet_id,
tablet_map_t& tablet_map = _get_tablet_map(tablet_id);
tablet_map[tablet_id] = tablet;
_add_tablet_to_partition(tablet);
+ // TODO: memory size of TabletSchema cannot be accurately tracked.
// TODO: remove multiply 2 of tablet meta mem size
// Because table schema will copy in tablet, there will be double mem cost
// so here multiply 2
- _tablet_meta_mem_tracker->consume(tablet->tablet_meta()->mem_size() * 2);
+ // _tablet_meta_schema_columns_tracker->consume(tablet->tablet_meta()->mem_size() * 2);
+ _tablet_meta_schema_columns_tracker->consume(tablet->tablet_meta()->tablet_columns_num());
Review Comment:
done，改成了bvar
##########
be/src/olap/tablet_manager.cpp:
##########
@@ -236,10 +234,12 @@ Status TabletManager::_add_tablet_to_map_unlocked(TTabletId tablet_id,
tablet_map_t& tablet_map = _get_tablet_map(tablet_id);
tablet_map[tablet_id] = tablet;
_add_tablet_to_partition(tablet);
+ // TODO: memory size of TabletSchema cannot be accurately tracked.
// TODO: remove multiply 2 of tablet meta mem size
// Because table schema will copy in tablet, there will be double mem cost
// so here multiply 2
- _tablet_meta_mem_tracker->consume(tablet->tablet_meta()->mem_size() * 2);
+ // _tablet_meta_schema_columns_tracker->consume(tablet->tablet_meta()->mem_size() * 2);
+ _tablet_meta_schema_columns_tracker->consume(tablet->tablet_meta()->tablet_columns_num());
Review Comment:
done，改成了bvar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1853365884
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1845880843
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.55 seconds
stream load tsv: 580 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17213739686 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1845863473
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit e2a20ab1682f2972a38fc688f63b21efde364a21, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4703 4432 4440 4432
q2 369 146 157 146
q3 1458 1264 1238 1238
q4 1122 905 883 883
q5 3148 3160 3112 3112
q6 249 131 128 128
q7 984 488 488 488
q8 2229 2213 2193 2193
q9 6656 6680 6682 6680
q10 3213 3242 3256 3242
q11 336 203 208 203
q12 359 210 212 210
q13 4542 3826 3817 3817
q14 242 207 219 207
q15 567 528 517 517
q16 444 390 388 388
q17 1012 561 513 513
q18 7401 8007 7052 7052
q19 1508 1326 1458 1326
q20 490 288 325 288
q21 3092 2623 2629 2623
q22 355 282 285 282
Total cold run time: 44479 ms
Total hot run time: 39968 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4402 4371 4369 4369
q2 265 164 174 164
q3 3540 3529 3512 3512
q4 2379 2368 2373 2368
q5 5733 5761 5747 5747
q6 241 122 121 121
q7 2393 1836 1914 1836
q8 3510 3509 3508 3508
q9 9040 9050 9037 9037
q10 3899 3998 3988 3988
q11 512 388 378 378
q12 755 607 608 607
q13 4317 3596 3554 3554
q14 280 253 252 252
q15 578 525 516 516
q16 507 489 478 478
q17 1865 1863 1866 1863
q18 8681 8205 8572 8205
q19 1741 1709 1727 1709
q20 2266 1950 1931 1931
q21 6522 6201 6173 6173
q22 516 439 417 417
Total cold run time: 63942 ms
Total hot run time: 60733 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1851846713
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1845704197
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1846491162
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1851963242
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 2caa6f991c3ccaf0e593aeb7602514a8fa2e28e6, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4677 4449 4490 4449
q2 366 138 174 138
q3 1467 1263 1269 1263
q4 1109 886 958 886
q5 3160 3158 3152 3152
q6 248 128 127 127
q7 1028 487 481 481
q8 2193 2233 2218 2218
q9 6714 6691 6684 6684
q10 3231 3265 3255 3255
q11 324 203 203 203
q12 352 206 213 206
q13 4567 3787 3786 3786
q14 239 220 211 211
q15 561 529 522 522
q16 442 399 384 384
q17 1016 647 542 542
q18 7720 7511 7016 7016
q19 1563 1479 1421 1421
q20 855 323 321 321
q21 3060 2611 2670 2611
q22 346 284 282 282
Total cold run time: 45238 ms
Total hot run time: 40158 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4425 4394 4428 4394
q2 266 169 176 169
q3 3538 3538 3525 3525
q4 2401 2391 2388 2388
q5 5753 5727 5756 5727
q6 241 125 120 120
q7 2380 1850 1890 1850
q8 3533 3528 3512 3512
q9 9057 8998 9025 8998
q10 3935 3983 3999 3983
q11 507 368 373 368
q12 766 592 600 592
q13 4326 3549 3597 3549
q14 292 255 261 255
q15 571 527 519 519
q16 497 482 465 465
q17 1873 1842 1835 1835
q18 8689 8524 8095 8095
q19 1769 1787 1746 1746
q20 2260 1930 1937 1930
q21 6560 6151 6162 6151
q22 505 438 404 404
Total cold run time: 64144 ms
Total hot run time: 60575 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1853365832
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #28149:
URL: https://github.com/apache/doris/pull/28149
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](memory) `TabletSchema` and `Schema` no longer track memory, only track columns count. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28149:
URL: https://github.com/apache/doris/pull/28149#issuecomment-1851981801
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.76 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.7 seconds inserted 10000000 Rows, about 336K ops/s
storage size: 17219660263 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org