You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "bobhan1 (via GitHub)" <gi...@apache.org> on 2023/11/17 07:46:14 UTC
[PR] [feature](merge-on-write) enable merge-on-write by default [doris]
bobhan1 opened a new pull request, #27188:
URL: https://github.com/apache/doris/pull/27188
## Proposed changes
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1816208190
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.81 seconds
stream load tsv: 583 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.8 seconds inserted 10000000 Rows, about 359K ops/s
storage size: 17098680756 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1816200671
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1816108051
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1820498448
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #27188:
URL: https://github.com/apache/doris/pull/27188
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1818227049
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.01 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.1 seconds inserted 10000000 Rows, about 355K ops/s
storage size: 17099573417 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1820795239
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1815885559
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1815902970
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1816204124
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1816269960
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.67 seconds
stream load tsv: 581 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17158456047 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1820884891
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1816080611
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.38 seconds
stream load tsv: 581 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17099940833 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1820884932
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1816156733
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.32 seconds
stream load tsv: 579 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17099699651 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1820696887
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit fc3adfa85180dd376129be3abc3ea0f53c67dbc5, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4890 4640 4649 4640
q2 359 155 160 155
q3 2021 1881 1902 1881
q4 1416 1252 1217 1217
q5 3915 3904 3987 3904
q6 241 128 132 128
q7 1425 888 910 888
q8 2736 2755 2747 2747
q9 57773 9561 9671 9561
q10 12987 3521 3518 3518
q11 372 241 250 241
q12 1679 289 290 289
q13 20478 3802 3784 3784
q14 315 290 285 285
q15 588 540 536 536
q16 664 574 580 574
q17 1136 991 941 941
q18 7790 7283 7349 7283
q19 1717 1688 1666 1666
q20 555 292 307 292
q21 7412 3957 3983 3957
q22 460 366 379 366
Total cold run time: 130929 ms
Total hot run time: 48853 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4672 4578 4571 4571
q2 339 232 278 232
q3 3988 3967 3969 3967
q4 2687 2668 2684 2668
q5 9753 9632 9637 9632
q6 238 119 120 119
q7 3002 2455 2474 2455
q8 4413 4438 4438 4438
q9 13145 13070 13139 13070
q10 4104 4180 4205 4180
q11 778 649 641 641
q12 971 800 829 800
q13 4266 3601 3622 3601
q14 374 332 362 332
q15 579 525 524 524
q16 728 691 676 676
q17 3841 3823 3931 3823
q18 9586 8973 8989 8973
q19 1827 1788 1769 1769
q20 2378 2032 2069 2032
q21 8826 8466 8384 8384
q22 885 799 792 792
Total cold run time: 81380 ms
Total hot run time: 77679 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1816113781
you can refer to my previous PR: #19017
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1816181145
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.41 seconds
stream load tsv: 578 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17100688470 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1816019710
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1818145212
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1820446654
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.77 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.3 seconds inserted 10000000 Rows, about 341K ops/s
storage size: 17098291592 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "zhannngchen (via GitHub)" <gi...@apache.org>.
zhannngchen commented on code in PR #27188:
URL: https://github.com/apache/doris/pull/27188#discussion_r1400553857
##########
docs/zh-CN/docs/data-table/data-model.md:
##########
@@ -416,7 +416,7 @@ mysql> select sum_merge(k2) , group_concat_merge(k3)from aggstate where k1 != 2;
在某些多维分析场景下,用户更关注的是如何保证 Key 的唯一性,即如何获得 Primary Key 唯一性约束。
因此,我们引入了 Unique 数据模型。在1.2版本之前,该模型本质上是聚合模型的一个特例,也是一种简化的表结构表示方式。
-由于聚合模型的实现方式是读时合并(merge on read),因此在一些聚合查询上性能不佳(参考后续章节[聚合模型的局限性](#聚合模型的局限性)的描述),
+由于聚合模型的实现方式是读时合并(merge on read),因此在一些聚合查询上性能不佳(参考后续章节[聚合模型的局限性](#聚合模型的局限性)的描述),
在1.2版本我们引入了Unique模型新的实现方式,写时合并(merge on write),通过在写入时做一些额外的工作,实现了最优的查询性能。
写时合并将在未来替换读时合并成为Unique模型的默认实现方式,两者将会短暂的共存一段时间。下面将对两种实现方式分别举例进行说明。
Review Comment:
“写时合并将在未来替换读时合并成为Unique模型的默认实现方式,两者将会短暂的共存一段时间”
update this description
##########
docs/zh-CN/docs/data-table/data-model.md:
##########
@@ -416,7 +416,7 @@ mysql> select sum_merge(k2) , group_concat_merge(k3)from aggstate where k1 != 2;
在某些多维分析场景下,用户更关注的是如何保证 Key 的唯一性,即如何获得 Primary Key 唯一性约束。
因此,我们引入了 Unique 数据模型。在1.2版本之前,该模型本质上是聚合模型的一个特例,也是一种简化的表结构表示方式。
-由于聚合模型的实现方式是读时合并(merge on read),因此在一些聚合查询上性能不佳(参考后续章节[聚合模型的局限性](#聚合模型的局限性)的描述),
+由于聚合模型的实现方式是读时合并(merge on read),因此在一些聚合查询上性能不佳(参考后续章节[聚合模型的局限性](#聚合模型的局限性)的描述),
在1.2版本我们引入了Unique模型新的实现方式,写时合并(merge on write),通过在写入时做一些额外的工作,实现了最优的查询性能。
写时合并将在未来替换读时合并成为Unique模型的默认实现方式,两者将会短暂的共存一段时间。下面将对两种实现方式分别举例进行说明。
Review Comment:
“写时合并将在未来替换读时合并成为Unique模型的默认实现方式,两者将会短暂的共存一段时间”
update this description
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1816265186
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.02 seconds
stream load tsv: 580 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17099983166 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1815999889
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1818230364
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit db91e487dd3a6749f3962cbb6653459a86aa9f5a, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4958 4687 4655 4655
q2 360 147 159 147
q3 2039 1976 1927 1927
q4 1395 1281 1242 1242
q5 3951 3943 4033 3943
q6 256 130 130 130
q7 1397 878 902 878
q8 2783 2782 2785 2782
q9 9899 9509 9343 9343
q10 3457 3519 3534 3519
q11 377 242 257 242
q12 440 300 291 291
q13 4574 3853 3825 3825
q14 320 293 282 282
q15 582 534 518 518
q16 671 586 589 586
q17 1132 923 920 920
q18 8005 7551 7371 7371
q19 1664 1683 1688 1683
q20 579 332 325 325
q21 4447 4035 4008 4008
q22 483 371 369 369
Total cold run time: 53769 ms
Total hot run time: 48986 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4582 4573 4589 4573
q2 330 223 247 223
q3 4010 4011 4026 4011
q4 2742 2725 2726 2725
q5 9625 9698 9611 9611
q6 252 121 125 121
q7 3025 2543 2504 2504
q8 4395 4417 4408 4408
q9 12991 12810 12987 12810
q10 4027 4158 4175 4158
q11 798 654 691 654
q12 971 818 824 818
q13 4297 3553 3534 3534
q14 391 346 353 346
q15 554 513 516 513
q16 750 683 670 670
q17 3916 3889 3894 3889
q18 9645 8908 9215 8908
q19 1777 1784 1774 1774
q20 2404 2034 2067 2034
q21 8870 8720 8724 8720
q22 882 826 812 812
Total cold run time: 81234 ms
Total hot run time: 77816 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1820379060
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1820486464
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 1bfadb1f21cb6166c4ebc13c6c00c30043f15215, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4913 4646 4652 4646
q2 369 144 140 140
q3 2028 1901 1910 1901
q4 1382 1253 1262 1253
q5 3929 3944 4018 3944
q6 240 132 132 132
q7 1413 897 890 890
q8 2740 2772 2756 2756
q9 18425 13553 9666 9666
q10 13171 3540 3521 3521
q11 394 254 247 247
q12 466 293 297 293
q13 21850 3784 3806 3784
q14 321 293 291 291
q15 590 539 527 527
q16 669 589 567 567
q17 1151 969 925 925
q18 7843 7269 7370 7269
q19 1663 1690 1673 1673
q20 535 306 308 306
q21 4435 3935 3937 3935
q22 464 362 366 362
Total cold run time: 88991 ms
Total hot run time: 49028 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4601 4576 4571 4571
q2 342 235 258 235
q3 4014 3994 3990 3990
q4 2707 2696 2713 2696
q5 9823 9687 9657 9657
q6 245 122 127 122
q7 2984 2457 2504 2457
q8 4440 4424 4397 4397
q9 13230 13122 13127 13122
q10 4101 4192 4216 4192
q11 813 711 648 648
q12 970 815 831 815
q13 4275 3598 3595 3595
q14 389 335 354 335
q15 572 520 531 520
q16 733 709 676 676
q17 3902 3892 3875 3875
q18 9510 9140 8896 8896
q19 1768 1782 1796 1782
q20 2411 2060 2033 2033
q21 8755 8622 8843 8622
q22 877 742 807 742
Total cold run time: 81462 ms
Total hot run time: 77978 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](merge-on-write) enable merge-on-write by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27188:
URL: https://github.com/apache/doris/pull/27188#issuecomment-1820705277
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.89 seconds
stream load tsv: 573 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17099339088 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org