You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yujun777 (via GitHub)" <gi...@apache.org> on 2023/06/25 01:50:49 UTC
[GitHub] [doris] yujun777 opened a new pull request, #21118: [Feature](x-load) support config min replica num for loading data
yujun777 opened a new pull request, #21118:
URL: https://github.com/apache/doris/pull/21118
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
Allows setting the minimum number of write replicas. When the number of successfully written copies exceeds it, the data import is successful.
1. If table's `min_load_replica_num` > 0, the minimum number of write replicas = table's `min_load_replica_num`;
2. Else if FE.conf's `min_load_replica_num` > 0, the minimum number of write replicas = `min(FE.conf.min_load_replica_num, table.replication_num/2+1)`；
3. Else the minimum number of write replicas = `table.replication_num/2+1`.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1606554955
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1748845313
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1606057790
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751589322
TeamCity be ut coverage result:
Function Coverage: 36.32% (8155/22453)
Line Coverage: 28.45% (65245/229308)
Region Coverage: 27.39% (33788/123363)
Branch Coverage: 24.05% (17232/71642)
Coverage Report: http://coverage.selectdb-in.cc/coverage/791ff8a85ba7c549624ac36e5ac0b6cfc4169d97_791ff8a85ba7c549624ac36e5ac0b6cfc4169d97/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #21118:
URL: https://github.com/apache/doris/pull/21118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751665495
TeamCity be ut coverage result:
Function Coverage: 36.31% (8155/22461)
Line Coverage: 28.44% (65245/229449)
Region Coverage: 27.38% (33798/123449)
Branch Coverage: 24.04% (17234/71698)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2e9ff6ac7a371fbbc94cff712ed770c8bd43217a_2e9ff6ac7a371fbbc94cff712ed770c8bd43217a/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1748858731
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1748935023
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.2 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162190216 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1748925163
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.25 seconds
stream load tsv: 556 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 35 seconds loaded 861443392 Bytes, about 23 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162079552 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1750988158
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.57 seconds
stream load tsv: 595 seconds loaded 74807831229 Bytes, about 119 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17162383131 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1733882122
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1750222074
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.69 seconds
stream load tsv: 563 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162201968 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1750387456
TeamCity be ut coverage result:
Function Coverage: 36.32% (8153/22450)
Line Coverage: 28.45% (65234/229278)
Region Coverage: 27.39% (33786/123358)
Branch Coverage: 24.05% (17233/71644)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5358f389240ed289e726b5981abc33a9ed8404fa_5358f389240ed289e726b5981abc33a9ed8404fa/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751583657
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751659526
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.75 seconds
stream load tsv: 557 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162317747 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1757317272
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1606000090
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1726783320
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1750355481
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751649939
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751651360
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1757343082
TeamCity be ut coverage result:
Function Coverage: 36.25% (8152/22489)
Line Coverage: 28.36% (65203/229887)
Region Coverage: 27.29% (33794/123830)
Branch Coverage: 23.98% (17214/71782)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b88c54a41dc725dc4362ced2e4996808d508983e_b88c54a41dc725dc4362ced2e4996808d508983e/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1757130132
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.62 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17162303394 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1726807076
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.25 seconds
stream load tsv: 601 seconds loaded 74807831229 Bytes, about 118 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17162537332 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1750061683
TeamCity be ut coverage result:
Function Coverage: 36.31% (8151/22450)
Line Coverage: 28.44% (65209/229278)
Region Coverage: 27.38% (33772/123358)
Branch Coverage: 24.03% (17219/71644)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4f7b3f99c1a96265dd431ef96626801100e8cc08_4f7b3f99c1a96265dd431ef96626801100e8cc08/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751661707
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751668029
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.4 seconds
stream load tsv: 554 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162248992 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1749998283
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1718753062
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.01 seconds
stream load tsv: 613 seconds loaded 74807831229 Bytes, about 116 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162366307 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #21118:
URL: https://github.com/apache/doris/pull/21118#discussion_r1339798437
##########
be/src/http/action/debug_point_action.cpp:
##########
@@ -43,32 +44,36 @@ void BaseDebugPointAction::handle(HttpRequest* req) {
}
Status AddDebugPointAction::_handle(HttpRequest* req) {
- std::string debug_point = req->param("debug_point");
+ std::string name = req->param("debug_point");
std::string execute = req->param("execute");
std::string timeout = req->param("timeout");
- if (debug_point.empty()) {
+ if (name.empty()) {
return Status::InternalError("Empty debug point name");
}
- int64_t execute_limit = -1;
- int64_t timeout_second = -1;
+ auto debug_point = std::make_shared<DebugPoint>();
try {
if (!execute.empty()) {
- execute_limit = std::stol(execute);
+ debug_point->execute_limit = std::stol(execute);
}
} catch (const std::exception& e) {
return Status::InternalError("Invalid execute limit format, execute {}, err {}", execute,
e.what());
}
try {
if (!timeout.empty()) {
- timeout_second = std::stol(timeout);
+ int64_t timeout_second = std::stol(timeout);
Review Comment:
warning: variable 'timeout_second' is not initialized [cppcoreguidelines-init-variables]
```suggestion
int64_t timeout_second = 0 = std::stol(timeout);
```
##########
be/src/http/action/debug_point_action.cpp:
##########
@@ -43,32 +44,36 @@ void BaseDebugPointAction::handle(HttpRequest* req) {
}
Status AddDebugPointAction::_handle(HttpRequest* req) {
- std::string debug_point = req->param("debug_point");
+ std::string name = req->param("debug_point");
Review Comment:
warning: variable 'name' is not initialized [cppcoreguidelines-init-variables]
```suggestion
std::string name = 0 = req->param("debug_point");
```
##########
be/src/util/debug_points.h:
##########
@@ -18,19 +18,21 @@
#pragma once
#include <atomic>
+#include <boost/lexical_cast.hpp>
Review Comment:
warning: 'boost/lexical_cast.hpp' file not found [clang-diagnostic-error]
```cpp
#include <boost/lexical_cast.hpp>
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1748906105
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.78 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162414048 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1750163630
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1750416142
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.75 seconds
stream load tsv: 563 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162332403 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1757062540
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1757371921
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.84 seconds
stream load tsv: 570 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17162366304 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1628066557
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1734001711
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.5 seconds
stream load tsv: 576 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17162131385 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1707792471
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1707812158
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1707850940
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.87 seconds
stream load tsv: 532 seconds loaded 74807831229 Bytes, about 134 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17161970282 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751656872
TeamCity be ut coverage result:
Function Coverage: 36.30% (8153/22461)
Line Coverage: 28.42% (65220/229449)
Region Coverage: 27.35% (33765/123449)
Branch Coverage: 24.02% (17224/71698)
Coverage Report: http://coverage.selectdb-in.cc/coverage/33e6c3c369e699cda06db47692c27727c920ae63_33e6c3c369e699cda06db47692c27727c920ae63/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1750974803
TeamCity be ut coverage result:
Function Coverage: 36.31% (8151/22450)
Line Coverage: 28.44% (65201/229278)
Region Coverage: 27.37% (33761/123358)
Branch Coverage: 24.04% (17222/71644)
Coverage Report: http://coverage.selectdb-in.cc/coverage/28ae5c3140d29be7d46eba7b0efd11677f3250b4_28ae5c3140d29be7d46eba7b0efd11677f3250b4/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on code in PR #21118:
URL: https://github.com/apache/doris/pull/21118#discussion_r1241119677
##########
docs/en/docs/data-operate/import/import-advanced/min-load-replica-num.md:
##########
@@ -0,0 +1,101 @@
+---
+{
+ "title": "Min Load Replica Num",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Min Load Replica Num
+
+Importing data requires more than half of the replicas to be written successfully. However, it is not flexible enough and may cause inconvenience in some scenarios.
+
+For example, in the case of two replicas, to import data, both replicas need to be written successfully. This means that no replica is allowed to be unavailable during the data import process. This greatly affects the availability of the cluster.
+
+In order to solve the above problems, Doris allows users to set the minimum number of write replicas. For the task of importing data, when the number of replicas it successfully writes is greater than or equal to the minimum number of replicas written, the import is successful.
+
+## Usage
+
+### Min load replica num for single table
+
+You can set the table property `min_load_replica_num` for a single olap table. The valid value of this property must be greater than 0 and not exceed `replication_num`（the number of replicas of the table）. Its default value is -1, indicating that the property is not enabled.
+
+The `min_load_replica_num` of the table can be set when creating the table.
+
+```sql
+CREATE TABLE test_table1
+(
+ k1 INT,
+ k2 INT
+)
+DUPLICATE KEY(k1)
+DISTRIBUTED BY HASH(k1) BUCKETS 5
+PROPERTIES
+(
+ 'replication_num' = '2',
+ 'min_load_replica_num' = '1'
+);
+```
+
+For an existing table, you can use `ALTER TABLE` to modify its `min_load_replica_num`.
+
+```sql
+ALTER TABLE test_table1
+SET ( 'min_load_replica_num' = '1');
+```
+
+You can use `SHOW CREATE TABLE` to view the table property `min_load_replica_num`.
+
+```SQL
+SHOW CREATE TABLE test_table1;
+```
+
+The PROPERTIES of the output will contain `min_load_replica_num`. e.g.
+
+```text
+Create Table: CREATE TABLE `test_table1` (
+ `k1` int(11) NULL,
+ `k2` int(11) NULL
+) ENGINE=OLAP
+DUPLICATE KEY(`k1`)
+COMMENT 'OLAP'
+DISTRIBUTED BY HASH(`k1`) BUCKETS 5
+PROPERTIES (
+"replication_allocation" = "tag.location.default: 2",
+"min_load_replica_num" = "1",
+"storage_format" = "V2",
+"light_schema_change" = "true",
+"disable_auto_compaction" = "false",
+"enable_single_replica_compaction" = "false"
+);
+```
+
+### Global min load replica num for all tables
+
+You can set FE configuration item `min_load_replica_num` for all olap tables. The valid value of this configuration item must be greater than 0. Its default value is -1, which means that the global minimum number of load replicas is not enabled.
+
+For a table, if the table property `min_load_replica_num` is valid (>0), then the table will ignore the global configuration `min_load_replica_num`. Otherwise, if the global configuration `min_load_replica_num` is valid (>0), then the minimum number of load replicas for the table will be equal to `min(FE.conf.min_load_replica_num, table.replication_num/2 + 1)`.
+
+For viewing and modification of FE configuration items, you can refer to [here](../../../admin-manual/config/fe-config.md).
+
+### Other cases
+
+If the table property `min_load_replica_num` is not enabled (<=0), and the global configuration `min_load_replica_num` is not enabled(<=0), then the data import still needs to be successfully written to the majority replica. At this point, the minimum number of write replicas for the table is equal to `table.replication_num/2 + 1`.
Review Comment:
add links in create table and fe-config. You'd better not put the doc and put content in create table and fe-config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1617148874
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1606347662
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1750197850
TeamCity be ut coverage result:
Function Coverage: 36.31% (8151/22450)
Line Coverage: 28.44% (65213/229278)
Region Coverage: 27.38% (33774/123358)
Branch Coverage: 24.04% (17224/71644)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cc6bd847fa06859222c7ac0042aaa05212170434_cc6bd847fa06859222c7ac0042aaa05212170434/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1748889230
TeamCity be ut coverage result:
Function Coverage: 36.32% (8153/22450)
Line Coverage: 28.45% (65233/229283)
Region Coverage: 27.38% (33780/123363)
Branch Coverage: 24.06% (17235/71648)
Coverage Report: http://coverage.selectdb-in.cc/coverage/719f1f5f5c36275d4aba9b65e0e94a99de174dcd_719f1f5f5c36275d4aba9b65e0e94a99de174dcd/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #21118:
URL: https://github.com/apache/doris/pull/21118#discussion_r1346974419
##########
be/src/util/debug_points.cpp:
##########
@@ -30,37 +30,42 @@ DebugPoints* DebugPoints::instance() {
}
bool DebugPoints::is_enable(const std::string& name) {
+ return get_debug_point(name) != nullptr;
+}
+
+std::shared_ptr<DebugPoint> DebugPoints::get_debug_point(const std::string& name) {
if (!config::enable_debug_points) {
- return false;
+ return nullptr;
}
auto map_ptr = std::atomic_load_explicit(&_debug_points, std::memory_order_relaxed);
auto it = map_ptr->find(name);
if (it == map_ptr->end()) {
- return false;
+ return nullptr;
}
- auto& debug_point = *(it->second);
- if ((debug_point.expire_ms > 0 && MonotonicMillis() >= debug_point.expire_ms) ||
- (debug_point.execute_limit > 0 &&
- debug_point.execute_num.fetch_add(1, std::memory_order_relaxed) >=
- debug_point.execute_limit)) {
+ auto debug_point = it->second;
+ if ((debug_point->expire_ms > 0 && MonotonicMillis() >= debug_point->expire_ms) ||
+ (debug_point->execute_limit > 0 &&
+ debug_point->execute_num.fetch_add(1, std::memory_order_relaxed) >=
+ debug_point->execute_limit)) {
remove(name);
- return false;
+ return nullptr;
}
- return true;
+ return debug_point;
}
-void DebugPoints::add(const std::string& name, int64_t execute_limit, int64_t timeout_second) {
- auto debug_point = std::make_shared<DebugPoint>();
- debug_point->execute_limit = execute_limit;
- if (timeout_second > 0) {
- debug_point->expire_ms = MonotonicMillis() + timeout_second * MILLIS_PER_SEC;
- }
+void DebugPoints::add(const std::string& name, std::shared_ptr<DebugPoint> debug_point) {
update([&](DebugPointMap& new_points) { new_points[name] = debug_point; });
- LOG(INFO) << "add debug point: name=" << name << ", execute=" << execute_limit
- << ", timeout=" << timeout_second;
+ std::ostringstream oss;
Review Comment:
warning: variable 'oss' is not initialized [cppcoreguidelines-init-variables]
```suggestion
std::ostringstream oss = 0;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1757637390
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1757637537
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751655323
TeamCity be ut coverage result:
Function Coverage: 36.31% (8155/22461)
Line Coverage: 28.44% (65244/229449)
Region Coverage: 27.36% (33779/123449)
Branch Coverage: 24.03% (17230/71698)
Coverage Report: http://coverage.selectdb-in.cc/coverage/33e6c3c369e699cda06db47692c27727c920ae63_33e6c3c369e699cda06db47692c27727c920ae63/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1613936414
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1606388723
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1718731506
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1748833786
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "yujun777 (via GitHub)" <gi...@apache.org>.
yujun777 commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1750875139
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751659740
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.42 seconds
stream load tsv: 557 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162292358 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1757100216
TeamCity be ut coverage result:
Function Coverage: 36.25% (8152/22489)
Line Coverage: 28.36% (65191/229879)
Region Coverage: 27.28% (33773/123806)
Branch Coverage: 23.98% (17209/71778)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b88c54a41dc725dc4362ced2e4996808d508983e_b88c54a41dc725dc4362ced2e4996808d508983e/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1750028245
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.68 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17161964431 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #21118:
URL: https://github.com/apache/doris/pull/21118#issuecomment-1751593750
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.21 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162180411 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org