You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "herry2038 (via GitHub)" <gi...@apache.org> on 2023/07/20 11:26:13 UTC
[GitHub] [doris] herry2038 opened a new pull request, #22040: [feature](show-frontends-disk) Add Show frontend disks
herry2038 opened a new pull request, #22040:
URL: https://github.com/apache/doris/pull/22040
## Proposed changes
Issue Number: close #xxx
Add feature show frontends disk which can show the disk information of fe 's interested dir, such as doris-meta, log, audit-log and temp.
As a example:
`
mysql> show frontends disks;
+-----------------------------------------+------------+-------------+-----------+-----------------------------+------------+----------+------+-----------+---------+---------+
| Name | Host | EditLogPort | DirType | Dir | Filesystem | Capacity | Used | Available | UseRate | MountOn |
+-----------------------------------------+------------+-------------+-----------+-----------------------------+------------+----------+------+-----------+---------+---------+
| fe_6a06ffaa_8346_4381_b462_8e33209ca6ec | 172.1.1.1 | 9010 | meta | E:\data\doris\fe/doris-meta | | 0 | 0 | 8191P | 0% | |
| fe_6a06ffaa_8346_4381_b462_8e33209ca6ec | 172.1.1.1 | 9010 | log | E:\data\doris\fe/log | | 0 | 0 | 8191P | 0% | |
| fe_6a06ffaa_8346_4381_b462_8e33209ca6ec | 172.1.1.1 | 9010 | audit-log | E:\data\doris\fe/log | | 0 | 0 | 8191P | 0% | |
| fe_6a06ffaa_8346_4381_b462_8e33209ca6ec | 172.1.1.1 | 9010 | temp | E:\data\doris\fe/temp_dir | | 0 | 0 | 8191P | 0% | |
| fe_fe1d5bd9_d1e5_4ccc_9b03_ca79b95c9941 | 172.1.1.2 | 9010 | meta | /data/doris/fe/doris-meta | /dev/sdc5 | 366G | 111G | 236G | 33% | /data |
| fe_fe1d5bd9_d1e5_4ccc_9b03_ca79b95c9941 | 172.1.1.2 | 9010 | log | /data/doris/fe/log | /dev/sdc5 | 366G | 111G | 236G | 33% | /data |
| fe_fe1d5bd9_d1e5_4ccc_9b03_ca79b95c9941 | 172.1.1.2 | 9010 | audit-log | /data/doris/fe/log | /dev/sdc5 | 366G | 111G | 236G | 33% | /data |
| fe_fe1d5bd9_d1e5_4ccc_9b03_ca79b95c9941 | 172.1.1.2 | 9010 | temp | /data/doris/fe/temp_dir | /dev/sdc5 | 366G | 111G | 236G | 33% | /data |
+-----------------------------------------+------------+-------------+-----------+-----------------------------+------------+----------+------+-----------+---------+---------+
8 rows in set (0.00 sec)
`
## Further comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1647288635
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1660115392
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1661696097
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] herry2038 commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "herry2038 (via GitHub)" <gi...@apache.org>.
herry2038 commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1660176301
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] herry2038 commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "herry2038 (via GitHub)" <gi...@apache.org>.
herry2038 commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1656576232
> Please add documentation. You can also add the tvf function if you are interested, ref: #20857
> Please add documentation. You can also add the tvf function if you are interested, ref: #20857
已经提交了文档了。tvf函数晚点在实现，也可以等下一个版本。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1660224746
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] herry2038 commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "herry2038 (via GitHub)" <gi...@apache.org>.
herry2038 commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1647112378
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] luozenglin merged pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "luozenglin (via GitHub)" <gi...@apache.org>.
luozenglin merged PR #22040:
URL: https://github.com/apache/doris/pull/22040
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] luozenglin commented on a diff in pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "luozenglin (via GitHub)" <gi...@apache.org>.
luozenglin commented on code in PR #22040:
URL: https://github.com/apache/doris/pull/22040#discussion_r1281452442
##########
fe/fe-core/src/main/java/org/apache/doris/service/ExecuteEnv.java:
##########
@@ -28,10 +33,55 @@ public class ExecuteEnv {
private ConnectScheduler scheduler;
private long startupTime;
+ public static class DiskInfo {
+ private String dirType;
Review Comment:
This class is used elsewhere and should be put in a class file, with `toThrift` and `fromThrift` as member method
##########
fe/fe-core/src/main/java/org/apache/doris/service/ExecuteEnv.java:
##########
@@ -56,4 +106,55 @@ public MultiLoadMgr getMultiLoadMgr() {
public long getStartupTime() {
return startupTime;
}
+
+ public List<DiskInfo> getDiskInfos() {
+ return this.diskInfos;
+ }
+
+ public List<DiskInfo> refreshAndGetDiskInfo(boolean refresh) {
+ for (DiskInfo disk : diskInfos) {
Review Comment:
Where do followers call this method to update the disk info?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] herry2038 commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "herry2038 (via GitHub)" <gi...@apache.org>.
herry2038 commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1655049540
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1645182498
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.39 seconds
stream load tsv: 506 seconds loaded 74807831229 Bytes, about 140 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162395358 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1663147833
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1660207848
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1661851194
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1656577177
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1655127195
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45 seconds
stream load tsv: 506 seconds loaded 74807831229 Bytes, about 140 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17161436661 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] herry2038 commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "herry2038 (via GitHub)" <gi...@apache.org>.
herry2038 commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1663149793
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] luozenglin commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "luozenglin (via GitHub)" <gi...@apache.org>.
luozenglin commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1652982579
Please add documentation. You can also add the tvf function if you are interested, ref: #20857
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1643995708
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1660189700
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1663174829
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.73 seconds
stream load tsv: 515 seconds loaded 74807831229 Bytes, about 138 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 29.8 seconds inserted 10000000 Rows, about 335K ops/s
storage size: 17162380869 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1663206462
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] herry2038 commented on a diff in pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "herry2038 (via GitHub)" <gi...@apache.org>.
herry2038 commented on code in PR #22040:
URL: https://github.com/apache/doris/pull/22040#discussion_r1277093196
##########
fe/fe-common/src/main/java/org/apache/doris/common/io/DiskUtils.java:
##########
@@ -0,0 +1,122 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common.io;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.List;
+
+public class DiskUtils {
+ public static class Df {
+ public String fileSystem = "";
+ public long blocks;
+ public long used;
+ public long available;
+ public int useRate;
+ public String mountedOn = "";
+ }
+
+ public static Df df(String dir) {
+ String os = System.getProperty("os.name");
+ if (os.startsWith("Windows")) {
+ Df df = new Df();
+ df.available = Long.MAX_VALUE / 1024;
+ return df;
+ }
+
+ Process process;
+ try {
+ process = Runtime.getRuntime().exec("df " + dir);
+ InputStream inputStream = process.getInputStream();
+ BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
+
+ // Filesystem 1K-blocks Used Available Use% Mounted on
+ // /dev/sdc 5814186096 5814169712 0 100% /home/spy-sd/sdc
+ String titleLine = reader.readLine();
+ String dataLine = reader.readLine();
+ if (titleLine == null || dataLine == null) {
+ return null;
+ }
+ String[] values = dataLine.split("\\s+");
+ if (values.length != 6) {
+ return null;
+ }
+
+ Df df = new Df();
+ df.fileSystem = values[0];
+ df.blocks = Long.parseLong(values[1]);
+ df.used = Long.parseLong(values[2]);
+ df.available = Long.parseLong(values[3]);
+ df.useRate = Integer.parseInt(values[4].replace("%", ""));
+ df.mountedOn = values[5];
+ return df;
+ } catch (IOException e) {
+ e.printStackTrace();
+ return null;
+ }
+ }
+
+ private static List<String> getTitles(String titlesLine) {
+ List<String> titles = new ArrayList<>();
+ String[] titleArray = titlesLine.split("\\s+");
+ for (String title : titleArray) {
+ if (title.equalsIgnoreCase("on")) {
+ if (!titles.isEmpty()) {
+ int lastIdx = titles.size() - 1;
+ titles.set(lastIdx, titles.get(lastIdx) + "On");
+ }
+ } else {
+ titles.add(title);
+ }
+ }
+ return titles;
+ }
+
+ private static String[] units = new String[]{"", "K", "M", "G", "T", "P"};
+
+ public static String sizeFormat(long size) {
+ int unitPos = 0;
+ while (size > 1024 && unitPos < units.length - 1) {
+ unitPos++;
+ size /= 1024;
+ }
+ return size + units[unitPos];
+ }
+
+ public static void main(String[] args) {
+ long [] values = new long[]{
+ 1L,
+ 1L * 1024,
+ 1L * 1024 * 1024,
+ 1L * 1024 * 1024 * 1024,
+ 1L * 1024 * 1024 * 1024 * 1024,
+ 1L * 1024 * 1024 * 1024 * 1024 * 1024,
+ 1L * 1024 * 1024 * 1024 * 1024 * 1024 * 1024,
+ };
+
+ for (long v : values) {
+ System.out.println(sizeFormat(v));
Review Comment:
Add a unit test class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] herry2038 commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "herry2038 (via GitHub)" <gi...@apache.org>.
herry2038 commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1645115901
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1643762029
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1661657200
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1645123507
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1655033636
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1662039622
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1663137331
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1663155989
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1643808554
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.61 seconds
stream load tsv: 505 seconds loaded 74807831229 Bytes, about 141 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 30.0 seconds inserted 10000000 Rows, about 333K ops/s
storage size: 17161954341 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1645052164
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.58 seconds
stream load tsv: 503 seconds loaded 74807831229 Bytes, about 141 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17161672703 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1662036745
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] herry2038 commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "herry2038 (via GitHub)" <gi...@apache.org>.
herry2038 commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1663133024
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] luozenglin commented on a diff in pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "luozenglin (via GitHub)" <gi...@apache.org>.
luozenglin commented on code in PR #22040:
URL: https://github.com/apache/doris/pull/22040#discussion_r1275780205
##########
fe/fe-common/src/main/java/org/apache/doris/common/io/DiskUtils.java:
##########
@@ -0,0 +1,122 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common.io;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.List;
+
+public class DiskUtils {
+ public static class Df {
+ public String fileSystem = "";
+ public long blocks;
+ public long used;
+ public long available;
+ public int useRate;
+ public String mountedOn = "";
+ }
+
+ public static Df df(String dir) {
+ String os = System.getProperty("os.name");
+ if (os.startsWith("Windows")) {
+ Df df = new Df();
+ df.available = Long.MAX_VALUE / 1024;
+ return df;
+ }
+
+ Process process;
+ try {
+ process = Runtime.getRuntime().exec("df " + dir);
+ InputStream inputStream = process.getInputStream();
+ BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
+
+ // Filesystem 1K-blocks Used Available Use% Mounted on
+ // /dev/sdc 5814186096 5814169712 0 100% /home/spy-sd/sdc
+ String titleLine = reader.readLine();
+ String dataLine = reader.readLine();
+ if (titleLine == null || dataLine == null) {
+ return null;
+ }
+ String[] values = dataLine.split("\\s+");
+ if (values.length != 6) {
+ return null;
+ }
+
+ Df df = new Df();
+ df.fileSystem = values[0];
+ df.blocks = Long.parseLong(values[1]);
+ df.used = Long.parseLong(values[2]);
+ df.available = Long.parseLong(values[3]);
+ df.useRate = Integer.parseInt(values[4].replace("%", ""));
+ df.mountedOn = values[5];
+ return df;
+ } catch (IOException e) {
+ e.printStackTrace();
+ return null;
+ }
+ }
+
+ private static List<String> getTitles(String titlesLine) {
+ List<String> titles = new ArrayList<>();
+ String[] titleArray = titlesLine.split("\\s+");
+ for (String title : titleArray) {
+ if (title.equalsIgnoreCase("on")) {
+ if (!titles.isEmpty()) {
+ int lastIdx = titles.size() - 1;
+ titles.set(lastIdx, titles.get(lastIdx) + "On");
+ }
+ } else {
+ titles.add(title);
+ }
+ }
+ return titles;
+ }
+
+ private static String[] units = new String[]{"", "K", "M", "G", "T", "P"};
+
+ public static String sizeFormat(long size) {
+ int unitPos = 0;
+ while (size > 1024 && unitPos < units.length - 1) {
+ unitPos++;
+ size /= 1024;
+ }
+ return size + units[unitPos];
+ }
+
+ public static void main(String[] args) {
+ long [] values = new long[]{
+ 1L,
+ 1L * 1024,
+ 1L * 1024 * 1024,
+ 1L * 1024 * 1024 * 1024,
+ 1L * 1024 * 1024 * 1024 * 1024,
+ 1L * 1024 * 1024 * 1024 * 1024 * 1024,
+ 1L * 1024 * 1024 * 1024 * 1024 * 1024 * 1024,
+ };
+
+ for (long v : values) {
+ System.out.println(sizeFormat(v));
Review Comment:
Is this for debugging? Please add unit tests or regression tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] herry2038 commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "herry2038 (via GitHub)" <gi...@apache.org>.
herry2038 commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1643751183
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1644856709
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] herry2038 commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "herry2038 (via GitHub)" <gi...@apache.org>.
herry2038 commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1645023777
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22040: [feature](show-frontends-disk) Add Show frontend disks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22040:
URL: https://github.com/apache/doris/pull/22040#issuecomment-1643995613
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org