You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Centurybbx (via GitHub)" <gi...@apache.org> on 2023/06/13 04:34:40 UTC
[GitHub] [doris] Centurybbx opened a new pull request, #20739: [Enhancement](config) optimize behavior of default_storage_medium
Centurybbx opened a new pull request, #20739:
URL: https://github.com/apache/doris/pull/20739
## Proposed changes
Issue Number: close #20345
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1633497111
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1633513076
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 50.58 seconds
stream load tsv: 508 seconds loaded 74807831229 Bytes, about 140 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17169816203 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230713120443_clickbench_pr_177815.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1627747565
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.83 seconds
stream load tsv: 438 seconds loaded 74807831229 Bytes, about 162 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 57 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 28 seconds loaded 861443392 Bytes, about 29 MB/s
insert into select: 65.1 seconds inserted 10000000 Rows, about 153K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230709152636_clickbench_pr_175294.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1643071786
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.59 seconds
stream load tsv: 524 seconds loaded 74807831229 Bytes, about 136 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.3 seconds inserted 10000000 Rows, about 341K ops/s
storage size: 17167547548 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1631748245
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.84 seconds
stream load tsv: 446 seconds loaded 74807831229 Bytes, about 159 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 29 seconds loaded 861443392 Bytes, about 28 MB/s
insert into select: 25.9 seconds inserted 10000000 Rows, about 386K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712021633_clickbench_pr_176783.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1631730311
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1628890277
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1642036162
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1628178105
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 53.92 seconds
stream load tsv: 503 seconds loaded 74807831229 Bytes, about 141 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 83.1 seconds inserted 10000000 Rows, about 120K ops/s
storage size: 17161113670 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230710125440_clickbench_pr_175622.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1631873836
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.91 seconds
stream load tsv: 506 seconds loaded 74807831229 Bytes, about 140 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17161932377 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712133016_clickbench_pr_176913.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1630954273
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1631017760
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 51.12 seconds
stream load tsv: 497 seconds loaded 74807831229 Bytes, about 143 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17161699548 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711231525_clickbench_pr_176647.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1641275594
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1631728464
run fe
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1631972405
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 50.4 seconds
stream load tsv: 457 seconds loaded 74807831229 Bytes, about 156 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 57 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 28 seconds loaded 861443392 Bytes, about 29 MB/s
insert into select: 26.5 seconds inserted 10000000 Rows, about 377K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712071243_clickbench_pr_176965.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1643654358
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1628550148
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1641300414
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.85 seconds
stream load tsv: 543 seconds loaded 74807831229 Bytes, about 131 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162427876 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring commented on a diff in pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on code in PR #20739:
URL: https://github.com/apache/doris/pull/20739#discussion_r1259739628
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##########
@@ -2954,4 +2952,54 @@ public long loadDb(DataInputStream dis, long checksum) throws IOException, DdlEx
public ConcurrentHashMap<Long, Database> getIdToDb() {
return new ConcurrentHashMap<>(idToDb);
}
+
+ private void createPartitionWithIndicesInternal(String clusterName, long dbId, long tableId, long baseIndexId,
Review Comment:
createPartitionWithIndicesElasticOnStorageMedium is a better name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on a diff in pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on code in PR #20739:
URL: https://github.com/apache/doris/pull/20739#discussion_r1259757160
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##########
@@ -2954,4 +2952,54 @@ public long loadDb(DataInputStream dis, long checksum) throws IOException, DdlEx
public ConcurrentHashMap<Long, Database> getIdToDb() {
return new ConcurrentHashMap<>(idToDb);
}
+
+ private void createPartitionWithIndicesInternal(String clusterName, long dbId, long tableId, long baseIndexId,
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1630957069
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1629992304
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 51.51 seconds
stream load tsv: 498 seconds loaded 74807831229 Bytes, about 143 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17167238907 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711100810_clickbench_pr_176128.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1628789366
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1643451707
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1643696012
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.73 seconds
stream load tsv: 509 seconds loaded 74807831229 Bytes, about 140 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.5 seconds inserted 10000000 Rows, about 338K ops/s
storage size: 17161160738 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1631928009
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] freemandealer commented on a diff in pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "freemandealer (via GitHub)" <gi...@apache.org>.
freemandealer commented on code in PR #20739:
URL: https://github.com/apache/doris/pull/20739#discussion_r1263282926
##########
fe/fe-core/src/test/java/org/apache/doris/catalog/ModifyStorageMediumTest.java:
##########
@@ -0,0 +1,64 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.common.DdlException;
+import org.apache.doris.system.Backend;
+import org.apache.doris.system.SystemInfoService;
+import org.apache.doris.thrift.TStorageMedium;
+import org.apache.doris.utframe.TestWithFeService;
+
+import org.junit.jupiter.api.Assertions;
+
+import java.util.List;
+
+public class ModifyStorageMediumTest extends TestWithFeService {
+
+ @Override
+ protected void runBeforeAll() throws Exception {
+ SystemInfoService clusterInfo = Env.getCurrentEnv().getClusterInfo();
+ List<Backend> allBackends = clusterInfo.getAllBackends();
+ // set all backends' storage medium to SSD
+ for (Backend backend : allBackends) {
+ if (backend.hasPathHash()) {
+ backend.getDisks().values().stream()
+ .peek(diskInfo -> diskInfo.setStorageMedium(TStorageMedium.SSD));
+ }
+ }
+ }
+
+ @Override
+ protected void runAfterAll() throws Exception {
+ Env.getCurrentEnv().clear();
+ }
+
+ public void modifyStorageMediumTest() throws Exception {
Review Comment:
what does the name `modifyStorageMediumTest` come from? you seems not `modify` anything?
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##########
@@ -2960,4 +2958,54 @@ public long loadDb(DataInputStream dis, long checksum) throws IOException, DdlEx
public ConcurrentHashMap<Long, Database> getIdToDb() {
return new ConcurrentHashMap<>(idToDb);
}
+
+ private void createPartitionWithIndicesElasticOnStorageMedium(String clusterName, long dbId, long tableId,
+ long baseIndexId, long partitionId, String partitionName, Map<Long, MaterializedIndexMeta> indexIdToMeta,
+ DistributionInfo distributionInfo, TStorageMedium storageMedium, ReplicaAllocation replicaAlloc,
+ Long versionInfo, Set<String> bfColumns, double bfFpp, Set<Long> tabletIdSet, List<Index> indexes,
+ boolean isInMemory, TStorageFormat storageFormat, TTabletType tabletType, TCompressionType compressionType,
+ DataSortInfo dataSortInfo, boolean enableUniqueKeyMergeOnWrite, String storagePolicy,
+ IdGeneratorBuffer idGeneratorBuffer, boolean disableAutoCompaction, boolean enableSingleReplicaCompaction,
+ boolean skipWriteIndexOnLoad, boolean storeRowColumn, boolean isDynamicSchema, BinlogConfig binlogConfig,
+ OlapTable olapTable, PartitionInfo partitionInfo, long bufferSize) throws DdlException {
+
+ try {
+ // create partition
+ Partition partition = createPartitionWithIndices(clusterName, dbId, tableId,
+ baseIndexId, partitionId, partitionName, indexIdToMeta, distributionInfo, storageMedium,
+ replicaAlloc, versionInfo, bfColumns, bfFpp, tabletIdSet, indexes, isInMemory, storageFormat,
+ tabletType, compressionType, dataSortInfo, enableUniqueKeyMergeOnWrite, storagePolicy,
+ idGeneratorBuffer, disableAutoCompaction, enableSingleReplicaCompaction, skipWriteIndexOnLoad,
+ storeRowColumn, isDynamicSchema, binlogConfig);
+ olapTable.addPartition(partition);
+ } catch (DdlException e) {
+ // check whether set storage medium from data property
Review Comment:
add comment here to tell we are using different medium because shortage on the original medium
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1589183350
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1627730043
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1628455615
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 51.18 seconds
stream load tsv: 498 seconds loaded 74807831229 Bytes, about 143 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 82.8 seconds inserted 10000000 Rows, about 120K ops/s
storage size: 17162235375 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230710161216_clickbench_pr_175791.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1630905094
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 53.49 seconds
stream load tsv: 501 seconds loaded 74807831229 Bytes, about 142 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17160504544 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711220922_clickbench_pr_176621.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1631958850
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 53.97 seconds
stream load tsv: 507 seconds loaded 74807831229 Bytes, about 140 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17168845826 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712150115_clickbench_pr_176967.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1638498391
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1642035546
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1640576680
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1640682023
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.26 seconds
stream load tsv: 511 seconds loaded 74807831229 Bytes, about 139 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17170620104 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1643030003
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1643768091
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #20739:
URL: https://github.com/apache/doris/pull/20739
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1631754734
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 50.83 seconds
stream load tsv: 499 seconds loaded 74807831229 Bytes, about 142 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size: 17161815627 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230712102715_clickbench_pr_176784.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx closed pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx closed pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
URL: https://github.com/apache/doris/pull/20739
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1631135569
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.21 seconds
stream load tsv: 444 seconds loaded 74807831229 Bytes, about 160 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 57 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 29 seconds loaded 861443392 Bytes, about 28 MB/s
insert into select: 25.9 seconds inserted 10000000 Rows, about 386K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711163027_clickbench_pr_176646.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1630957166
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on a diff in pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on code in PR #20739:
URL: https://github.com/apache/doris/pull/20739#discussion_r1263294809
##########
fe/fe-core/src/test/java/org/apache/doris/catalog/ModifyStorageMediumTest.java:
##########
@@ -0,0 +1,64 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.common.DdlException;
+import org.apache.doris.system.Backend;
+import org.apache.doris.system.SystemInfoService;
+import org.apache.doris.thrift.TStorageMedium;
+import org.apache.doris.utframe.TestWithFeService;
+
+import org.junit.jupiter.api.Assertions;
+
+import java.util.List;
+
+public class ModifyStorageMediumTest extends TestWithFeService {
+
+ @Override
+ protected void runBeforeAll() throws Exception {
+ SystemInfoService clusterInfo = Env.getCurrentEnv().getClusterInfo();
+ List<Backend> allBackends = clusterInfo.getAllBackends();
+ // set all backends' storage medium to SSD
+ for (Backend backend : allBackends) {
+ if (backend.hasPathHash()) {
+ backend.getDisks().values().stream()
+ .peek(diskInfo -> diskInfo.setStorageMedium(TStorageMedium.SSD));
+ }
+ }
+ }
+
+ @Override
+ protected void runAfterAll() throws Exception {
+ Env.getCurrentEnv().clear();
+ }
+
+ public void modifyStorageMediumTest() throws Exception {
Review Comment:
That's a problem, I will modify it and add some new tests later!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1596164051
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Centurybbx commented on pull request #20739: [Enhancement](config) optimize behavior of default_storage_medium
Posted by "Centurybbx (via GitHub)" <gi...@apache.org>.
Centurybbx commented on PR #20739:
URL: https://github.com/apache/doris/pull/20739#issuecomment-1596137863
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org