You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ozone.apache.org by "Elek, Marton" <el...@apache.org> on 2021/04/06 12:40:28 UTC
Fwd: I have a problem with Ozone 1.0.0
-------- Forwarded Message --------
Subject: I have a problem with Ozone 1.0.0
Date: Tue, 6 Apr 2021 12:16:57 +0800
From: dongyanhui@nnct-nsn.com <do...@nnct-nsn.com>
To: issues <is...@ozone.apache.org>
Hi,Support!
1：I have installed the Ozone 1.0.1 sucessfully with 5 datanodes.
[root@node221 logs]# ozone admin datanode list
Datanode: 3e970fc5-c0a3-46c0-812f-9743ea879fce (/default-rack/192.168.21.226/node226.cluster1.com/1 pipelines)
Related pipelines:
9004918c-6026-49a4-a024-5477cef464ab/ONE/RATIS/OPEN/Leader
Datanode: e18fd3bf-009f-4d45-a358-4a8efdd4ef30 (/default-rack/192.168.21.221/node221.cluster1.com/1 pipelines)
Related pipelines:
e75d2266-2aa1-49ad-ac16-0ed6895c9550/ONE/RATIS/OPEN/Leader
Datanode: d07c8e6b-566f-4b5d-a21c-648a053c5b45 (/default-rack/192.168.21.220/node220.cluster1.com/2 pipelines)
Related pipelines:
5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Follower
e000dffa-8a70-4833-9a9b-eb13e367e7dd/ONE/RATIS/OPEN/Leader
Datanode: 951aef66-9b85-4942-a218-a25f758f192d (/default-rack/192.168.21.219/node219.cluster1.com/2 pipelines)
Related pipelines:
5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Follower
54d792ec-5117-4678-951e-e22ca99bf5e5/ONE/RATIS/OPEN/Leader
Datanode: f3e011c6-e08e-4aba-8b43-dc4bfa87f752 (/default-rack/192.168.21.225/node225.cluster1.com/2 pipelines)
Related pipelines:
3f6d26d2-52c2-41f9-8e8f-ede7c111074f/ONE/RATIS/OPEN/Leader
5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Leader
2：I have a problem with Ozone 1.0.0 as bellow:
The dns can increase more than 3 nodes, but The data can upload to 3
datanodes only,*is their licensing restrictions?*
[root@node219 hadoop]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
4.0K /data01/ozone/db.checkpoints
7.4G /data01/ozone/dn01
7.2G /data01/ozone/dn02
7.7G /data01/ozone/dn03
12K /data01/ozone/om
146M /data01/ozone/om.db
4.0K /data01/ozone/omMetrics
6.6M /data01/ozone/ratis
12K /data01/ozone/scm
146M /data01/ozone/scm.db
[root@node220 ~]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
7.5G /data01/ozone/dn01
7.3G /data01/ozone/dn02
7.7G /data01/ozone/dn03
6.7M /data01/ozone/ratis
[root@node221 logs]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
20K /data01/ozone/dn01
20K /data01/ozone/dn02
20K /data01/ozone/dn03
48K /data01/ozone/ratis
[root@node225 ~]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
7.7G /data01/ozone/dn01
7.3G /data01/ozone/dn02
7.9G /data01/ozone/dn03
6.7M /data01/ozone/ratis
[root@node226 ~]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
16K /data01/ozone/dn01
16K /data01/ozone/dn02
16K /data01/ozone/dn03
44K /data01/ozone/ratis
3: The ozone config as bellow:
[root@node219 hadoop]# cat ozone-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
<property>
<name>ozone.om.address</name>
<value>192.168.21.219</value>
<tag>OM, REQUIRED</tag>
<description>
The address of the Ozone OM service. This allows clients to discover
the address of the OM.
</description>
</property>
<property>
<name>ozone.metadata.dirs</name>
<value>/data01/ozone</value>
<tag>OZONE, OM, SCM, CONTAINER, STORAGE, REQUIRED</tag>
<description>
This setting is the fallback location for SCM, OM, Recon and DataNodes
to store their metadata. This setting may be used only in test/PoC
clusters to simplify configuration.
For production clusters or any time you care about performance, it is
recommended that ozone.om.db.dirs, ozone.scm.db.dirs and
dfs.container.ratis.datanode.storage.dir be configured separately.
</description>
</property>
<property>
<name>ozone.scm.client.address</name>
<value>192.168.21.219</value>
<tag>OZONE, SCM, REQUIRED</tag>
<description>
The address of the Ozone SCM client service. This is a required setting.
It is a string in the host:port format. The port number is optional
and defaults to 9860.
</description>
</property>
<property>
<name>ozone.scm.names</name>
<value>192.168.21.219</value>
<tag>OZONE, REQUIRED</tag>
<description>
The value of this property is a set of DNS | DNS:PORT | IP
Address | IP:PORT. Written as a comma separated string. e.g. scm1,
scm2:8020, 7.7.7.7:7777.
This property allows datanodes to discover where SCM is, so that
datanodes can send heartbeat to SCM.
</description>
</property>
<property>
<name>hdds.datanode.dir</name>
<value>/data01/ozone/dn01,/data01/ozone/dn02,/data01/ozone/dn03</value>
<tag>OZONE, CONTAINER, STORAGE, MANAGEMENT</tag>
<description>Determines where on the local filesystem HDDS data will be
stored. Defaults to dfs.datanode.data.dir if not specified.
The directories should be tagged with corresponding storage types
([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for storage policies. The default
storage type will be DISK if the directory does not have a storage type
tagged explicitly.
</description>
</property>
</configuration>
4: ozone version
[root@node221 logs]# ozone version
//////////////
////////////////////
//////// ////////////////
////// ////////////////
///// //////////////// /
///// //////// ///
//// //////// /////
///// ////////////////
///// //////////////// //
//// /////////////// /////
///// /////////////// ////
///// ////// /////
////// ////// /////
/////////// ////////
////// ////////////
/// //////////
/ 1.0.0(Denali)
Source code repository Unknown -r Unknown
Compiled by root on 2021-04-02T09:53Z
Compiled with protoc 2.5.0
From source with checksum 49ea4ac8358b9bfbdcda09565e1fcd8
Using HDDS 1.0.0
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Re: I have a problem with Ozone 1.0.0
Posted by "dongyanhui@nnct-nsn.com" <do...@nnct-nsn.com>.
It works well now,Thank you very much :)
[root@node219 ~]# ozone admin datanode list
Datanode: b3aecba7-f55e-4808-b6d5-c1868f4231be (/default-rack/192.168.21.225/node225.cluster1.com/2 pipelines)
Related pipelines:
6843513d-8a94-41d7-9654-f94f65618e77/THREE/RATIS/OPEN/Follower
7ef266d6-9cf7-4f69-b1f2-be79cc2b0c97/THREE/RATIS/OPEN/Follower
Datanode: 04f7e902-257d-46d2-b48f-f8fc9e4e0956 (/default-rack/192.168.21.226/node226.cluster1.com/2 pipelines)
Related pipelines:
6843513d-8a94-41d7-9654-f94f65618e77/THREE/RATIS/OPEN/Follower
7ef266d6-9cf7-4f69-b1f2-be79cc2b0c97/THREE/RATIS/OPEN/Follower
Datanode: 6e1b8862-74d0-42f6-82dc-d36ed7e47ff8 (/default-rack/192.168.21.227/node227.cluster1.com/2 pipelines)
Related pipelines:
6843513d-8a94-41d7-9654-f94f65618e77/THREE/RATIS/OPEN/Leader
7ef266d6-9cf7-4f69-b1f2-be79cc2b0c97/THREE/RATIS/OPEN/Leader
Datanode: ea062c92-bc28-4465-bf7a-0bc3638e5fda (/default-rack/192.168.21.222/node222.cluster1.com/2 pipelines)
Related pipelines:
d7a6361a-2f64-4015-a076-41c6f22980bd/THREE/RATIS/OPEN/Follower
8b3f3cc6-138b-4760-aa67-f750cc706898/THREE/RATIS/OPEN/Follower
Datanode: fd803946-6332-4517-8257-58d39baccbf1 (/default-rack/192.168.21.223/node223.cluster1.com/2 pipelines)
Related pipelines:
d7a6361a-2f64-4015-a076-41c6f22980bd/THREE/RATIS/OPEN/Follower
8b3f3cc6-138b-4760-aa67-f750cc706898/THREE/RATIS/OPEN/Follower
Datanode: 6e6e488c-b974-4c0f-a09d-f8c6f515706a (/default-rack/192.168.21.224/node224.cluster1.com/2 pipelines)
Related pipelines:
d7a6361a-2f64-4015-a076-41c6f22980bd/THREE/RATIS/OPEN/Leader
8b3f3cc6-138b-4760-aa67-f750cc706898/THREE/RATIS/OPEN/Leader
[root@node222 ~]# du -sh /data01/ozone/* |grep dn
1.7G /data01/ozone/dn01
[root@node223 ~]# du -sh /data01/ozone/* |grep dn
1.7G /data01/ozone/dn01
[root@node224 ~]# du -sh /data01/ozone/* |grep dn
1.7G /data01/ozone/dn01
[root@node225 ~]# du -sh /data01/ozone/* |grep dn
1.4G /data01/ozone/dn01
[root@node226 ~]# du -sh /data01/ozone/* |grep dn
1.4G /data01/ozone/dn01
[root@node227 ~]# du -sh /data01/ozone/* |grep dn
1.4G /data01/ozone/dn01
此致，敬礼！
董延辉
电话：13633860082
Email:dongyanhui@nnct-nsn.com
From: dongyanhui@nnct-nsn.com
Date: 2021-04-08 10:51
To: Sammi Chen
CC: dev
Subject: Re: Re: I have a problem with Ozone 1.0.0
Hi,Sammi
1: When i changed two values of the "ozone-site.xml" as bellow :
ozone.replication=1
ozone.scm.pipeline.creation.auto.factor.one=false
2: Ther Ozone cluster seems two nodes(192.168.21.225,192.168.21.225) were Unhealthy.....
Is the "Ozone 1.0.0" version's license limit 3 datanodes only?
[root@node219 hadoop]# ozone admin datanode list
Datanode: 81ef6354-82cf-4986-93b4-de8d7cff9a55 (/default-rack/192.168.21.221/node221.cluster1.com/2 pipelines)
Related pipelines:
601532ff-53e9-4576-a83c-b6d103aa2e46/THREE/RATIS/OPEN/Follower
d12d4996-6d0f-4a9e-b9c9-7aed68bd33e8/THREE/RATIS/OPEN/Leader
Datanode: 38fd5463-e967-4bcb-b821-9d9043c8da59 (/default-rack/192.168.21.225/node225.cluster1.com/0 pipelines)
Related pipelines:
No related pipelines or the node is not in Healthy state.
Datanode: 952597ab-c148-4197-94e2-156ecce533fe (/default-rack/192.168.21.226/node226.cluster1.com/0 pipelines)
Related pipelines:
No related pipelines or the node is not in Healthy state.
Datanode: cfc00f4d-f3c1-4bcb-b66a-1f0a4716b368 (/default-rack/192.168.21.219/node219.cluster1.com/2 pipelines)
Related pipelines:
601532ff-53e9-4576-a83c-b6d103aa2e46/THREE/RATIS/OPEN/Follower
d12d4996-6d0f-4a9e-b9c9-7aed68bd33e8/THREE/RATIS/OPEN/Follower
Datanode: 6f13abae-9f7c-4c6d-85f4-4859fb474519 (/default-rack/192.168.21.220/node220.cluster1.com/2 pipelines)
Related pipelines:
601532ff-53e9-4576-a83c-b6d103aa2e46/THREE/RATIS/OPEN/Leader
d12d4996-6d0f-4a9e-b9c9-7aed68bd33e8/THREE/RATIS/OPEN/Follower
3: config files
[root@node219 hadoop]# cat ozone-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
<property>
<name>ozone.om.address</name>
<value>node219.cluster1.com</value>
<tag>OM, REQUIRED</tag>
<description>
The address of the Ozone OM service. This allows clients to discover
the address of the OM.
</description>
</property>
<property>
<name>ozone.metadata.dirs</name>
<value>/data01/ozone</value>
<tag>OZONE, OM, SCM, CONTAINER, STORAGE, REQUIRED</tag>
<description>
This setting is the fallback location for SCM, OM, Recon and DataNodes
to store their metadata. This setting may be used only in test/PoC
clusters to simplify configuration.
For production clusters or any time you care about performance, it is
recommended that ozone.om.db.dirs, ozone.scm.db.dirs and
dfs.container.ratis.datanode.storage.dir be configured separately.
</description>
</property>
<property>
<name>ozone.scm.client.address</name>
<value>node219.cluster1.com</value>
<tag>OZONE, SCM, REQUIRED</tag>
<description>
The address of the Ozone SCM client service. This is a required setting.
It is a string in the host:port format. The port number is optional
and defaults to 9860.
</description>
</property>
<property>
<name>ozone.scm.names</name>
<value>node219.cluster1.com</value>
<tag>OZONE, REQUIRED</tag>
<description>
The value of this property is a set of DNS | DNS:PORT | IP
Address | IP:PORT. Written as a comma separated string. e.g. scm1,
scm2:8020, 7.7.7.7:7777.
This property allows datanodes to discover where SCM is, so that
datanodes can send heartbeat to SCM.
</description>
</property>
<property>
<name>hdds.datanode.dir</name>
<value>/data01/ozone/dn01</value>
<tag>OZONE, CONTAINER, STORAGE, MANAGEMENT</tag>
<description>Determines where on the local filesystem HDDS data will be
stored. Defaults to dfs.datanode.data.dir if not specified.
The directories should be tagged with corresponding storage types
([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for storage policies. The default
storage type will be DISK if the directory does not have a storage type
tagged explicitly.
</description>
</property>
<property>
<name>ozone.scm.heartbeat.rpc-timeout</name>
<value>30s</value>
<tag>OZONE, MANAGEMENT</tag>
<description>
Timeout value for the RPC from Datanode to SCM.
</description>
</property>
<property>
<name>ozone.scm.heartbeat.rpc-retry-interval</name>
<value>60s</value>
<tag>OZONE, MANAGEMENT</tag>
<description>
Retry interval for the RPC from Datanode to SCM.
Make sure rpc-retry-count * (rpc-timeout + rpc-retry-interval)
is less than hdds.heartbeat.interval.
</description>
</property>
<property>
<name>ozone.scm.pipeline.creation.auto.factor.one</name>
<value>false</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>
If enabled, SCM will auto create RATIS factor ONE pipeline.
</description>
</property>
<property>
<name>ozone.scm.datanode.pipeline.limit</name>
<value>6</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>Max number of pipelines per datanode can be engaged in.
Setting the value to 0 means the pipeline limit per dn will be determined
by the no of metadata volumes reported per dn.
</description>
</property>
<property>
<name>ozone.scm.db.dirs</name>
<value>/data01/ozone</value>
<tag>OZONE, OM, SCM, CONTAINER, STORAGE, REQUIRED</tag>
<description>
This setting is the fallback location for SCM, OM, Recon and DataNodes
to store their metadata. This setting may be used only in test/PoC
clusters to simplify configuration.
For production clusters or any time you care about performance, it is
recommended that ozone.om.db.dirs, ozone.scm.db.dirs and
dfs.container.ratis.datanode.storage.dir be configured separately.
</description>
</property>
<property>
<name>hdds.node.report.interval</name>
<value>60000ms</value>
<tag>OZONE, CONTAINER, MANAGEMENT</tag>
<description>Time interval of the datanode to send node report. Each
datanode periodically send node report to SCM. Unit could be
defined with postfix (ns,ms,s,m,h,d)</description>
</property>
<property>
<name>ozone.replication</name>
<value>1</value>
<tag>OZONE, CLIENT</tag>
<description>
Default replication value. The actual number of replications can be
specified when writing the key. The default is used if replication
is not specified. Supported values: 1 and 3.
</description>
</property>
</configuration>
此致，敬礼！
董延辉
电话：13633860082
Email:dongyanhui@nnct-nsn.com
From: dongyanhui@nnct-nsn.com
Date: 2021-04-07 14:06
To: Sammi Chen
CC: dev
Subject: Re: Re: I have a problem with Ozone 1.0.0
Yes, i configed it for all nodes in thes cluster,and reboot all services ,but the same problem.
此致，敬礼！
董延辉
电话：13633860082
Email:dongyanhui@nnct-nsn.com
From: Sammi Chen
Date: 2021-04-07 13:40
To: dongyanhui@nnct-nsn.com
CC: dev
Subject: Re: Re: I have a problem with Ozone 1.0.0
These properties are required on both SCM and DN. And a cluster reboot is required.
On Wed, Apr 7, 2021 at 11:44 AM dongyanhui@nnct-nsn.com <do...@nnct-nsn.com> wrote:
Thank you for update:)
The same issue when i configed the parameters you give me.
[root@node219 hadoop]# cat ozone-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
<property>
<name>ozone.om.address</name>
<value>192.168.21.219</value>
<tag>OM, REQUIRED</tag>
<description>
The address of the Ozone OM service. This allows clients to discover
the address of the OM.
</description>
</property>
<property>
<name>ozone.metadata.dirs</name>
<value>/data01/ozone</value>
<tag>OZONE, OM, SCM, CONTAINER, STORAGE, REQUIRED</tag>
<description>
This setting is the fallback location for SCM, OM, Recon and DataNodes
to store their metadata. This setting may be used only in test/PoC
clusters to simplify configuration.
For production clusters or any time you care about performance, it is
recommended that ozone.om.db.dirs, ozone.scm.db.dirs and
dfs.container.ratis.datanode.storage.dir be configured separately.
</description>
</property>
<property>
<name>ozone.scm.client.address</name>
<value>192.168.21.219</value>
<tag>OZONE, SCM, REQUIRED</tag>
<description>
The address of the Ozone SCM client service. This is a required setting.
It is a string in the host:port format. The port number is optional
and defaults to 9860.
</description>
</property>
<property>
<name>ozone.scm.names</name>
<value>192.168.21.219</value>
<tag>OZONE, REQUIRED</tag>
<description>
The value of this property is a set of DNS | DNS:PORT | IP
Address | IP:PORT. Written as a comma separated string. e.g. scm1,
scm2:8020, 7.7.7.7:7777.
This property allows datanodes to discover where SCM is, so that
datanodes can send heartbeat to SCM.
</description>
</property>
<property>
<name>hdds.datanode.dir</name>
<value>/data01/ozone/dn01</value>
<tag>OZONE, CONTAINER, STORAGE, MANAGEMENT</tag>
<description>Determines where on the local filesystem HDDS data will be
stored. Defaults to dfs.datanode.data.dir if not specified.
The directories should be tagged with corresponding storage types
([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for storage policies. The default
storage type will be DISK if the directory does not have a storage type
tagged explicitly.
</description>
</property>
<property>
<name>ozone.scm.heartbeat.rpc-timeout</name>
<value>30s</value>
<tag>OZONE, MANAGEMENT</tag>
<description>
Timeout value for the RPC from Datanode to SCM.
</description>
</property>
<property>
<name>ozone.scm.heartbeat.rpc-retry-interval</name>
<value>60s</value>
<tag>OZONE, MANAGEMENT</tag>
<description>
Retry interval for the RPC from Datanode to SCM.
Make sure rpc-retry-count * (rpc-timeout + rpc-retry-interval)
is less than hdds.heartbeat.interval.
</description>
</property>
<property>
<name>ozone.scm.pipeline.creation.auto.factor.one</name>
<value>true</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>
If enabled, SCM will auto create RATIS factor ONE pipeline.
</description>
</property>
<property>
<name>ozone.scm.datanode.pipeline.limit</name>
<value>5</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>Max number of pipelines per datanode can be engaged in.
Setting the value to 0 means the pipeline limit per dn will be determined
by the no of metadata volumes reported per dn.
</description>
</property>
</configuration>
此致，敬礼！
董延辉
电话：13633860082
Email:dongyanhui@nnct-nsn.com
From: Sammi Chen
Date: 2021-04-07 10:22
To: dev
CC: dongyanhui
Subject: Re: I have a problem with Ozone 1.0.0
Hey dongyanhui，
From the " ozone admin datanode list " output, we can see that there is only one THREE factor pipeline, which involves three datanode 219, 220, 225. I guess your data requires three replicas, so that your data is written to these three datanodes.
To tuning the cluster, you can try,
1. controls whether factor one pipeline should be created. If you don't need to write one replica data, you can turn this off .
<property>
<name>ozone.scm.pipeline.creation.auto.factor.one</name>
<value>true</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>
If enabled, SCM will auto create RATIS factor ONE pipeline.
</description>
</property>
2. controls how many pipelines can a DN have. You can increase this number, say 5, if you have more than 5 DN disks configured.
<property>
<name>ozone.scm.datanode.pipeline.limit</name>
<value>2</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>Max number of pipelines per datanode can be engaged in.
Setting the value to 0 means the pipeline limit per dn will be determined
by the no of metadata volumes reported per dn.
</description>
</property>
Regards,
Sammi
On Tue, Apr 6, 2021 at 8:41 PM Elek, Marton <el...@apache.org> wrote:
-------- Forwarded Message --------
Subject: I have a problem with Ozone 1.0.0
Date: Tue, 6 Apr 2021 12:16:57 +0800
From: dongyanhui@nnct-nsn.com <do...@nnct-nsn.com>
To: issues <is...@ozone.apache.org>
Hi,Support!
1：I have installed the Ozone 1.0.1 sucessfully with 5 datanodes.
[root@node221 logs]# ozone admin datanode list
Datanode: 3e970fc5-c0a3-46c0-812f-9743ea879fce (/default-rack/192.168.21.226/node226.cluster1.com/1 pipelines)
Related pipelines:
9004918c-6026-49a4-a024-5477cef464ab/ONE/RATIS/OPEN/Leader
Datanode: e18fd3bf-009f-4d45-a358-4a8efdd4ef30 (/default-rack/192.168.21.221/node221.cluster1.com/1 pipelines)
Related pipelines:
e75d2266-2aa1-49ad-ac16-0ed6895c9550/ONE/RATIS/OPEN/Leader
Datanode: d07c8e6b-566f-4b5d-a21c-648a053c5b45 (/default-rack/192.168.21.220/node220.cluster1.com/2 pipelines)
Related pipelines:
5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Follower
e000dffa-8a70-4833-9a9b-eb13e367e7dd/ONE/RATIS/OPEN/Leader
Datanode: 951aef66-9b85-4942-a218-a25f758f192d (/default-rack/192.168.21.219/node219.cluster1.com/2 pipelines)
Related pipelines:
5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Follower
54d792ec-5117-4678-951e-e22ca99bf5e5/ONE/RATIS/OPEN/Leader
Datanode: f3e011c6-e08e-4aba-8b43-dc4bfa87f752 (/default-rack/192.168.21.225/node225.cluster1.com/2 pipelines)
Related pipelines:
3f6d26d2-52c2-41f9-8e8f-ede7c111074f/ONE/RATIS/OPEN/Leader
5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Leader
2：I have a problem with Ozone 1.0.0 as bellow:
The dns can increase more than 3 nodes, but The data can upload to 3
datanodes only,*is their licensing restrictions?*
[root@node219 hadoop]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
4.0K /data01/ozone/db.checkpoints
7.4G /data01/ozone/dn01
7.2G /data01/ozone/dn02
7.7G /data01/ozone/dn03
12K /data01/ozone/om
146M /data01/ozone/om.db
4.0K /data01/ozone/omMetrics
6.6M /data01/ozone/ratis
12K /data01/ozone/scm
146M /data01/ozone/scm.db
[root@node220 ~]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
7.5G /data01/ozone/dn01
7.3G /data01/ozone/dn02
7.7G /data01/ozone/dn03
6.7M /data01/ozone/ratis
[root@node221 logs]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
20K /data01/ozone/dn01
20K /data01/ozone/dn02
20K /data01/ozone/dn03
48K /data01/ozone/ratis
[root@node225 ~]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
7.7G /data01/ozone/dn01
7.3G /data01/ozone/dn02
7.9G /data01/ozone/dn03
6.7M /data01/ozone/ratis
[root@node226 ~]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
16K /data01/ozone/dn01
16K /data01/ozone/dn02
16K /data01/ozone/dn03
44K /data01/ozone/ratis
3: The ozone config as bellow:
[root@node219 hadoop]# cat ozone-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
<property>
<name>ozone.om.address</name>
<value>192.168.21.219</value>
<tag>OM, REQUIRED</tag>
<description>
The address of the Ozone OM service. This allows clients to discover
the address of the OM.
</description>
</property>
<property>
<name>ozone.metadata.dirs</name>
<value>/data01/ozone</value>
<tag>OZONE, OM, SCM, CONTAINER, STORAGE, REQUIRED</tag>
<description>
This setting is the fallback location for SCM, OM, Recon and DataNodes
to store their metadata. This setting may be used only in test/PoC
clusters to simplify configuration.
For production clusters or any time you care about performance, it is
recommended that ozone.om.db.dirs, ozone.scm.db.dirs and
dfs.container.ratis.datanode.storage.dir be configured separately.
</description>
</property>
<property>
<name>ozone.scm.client.address</name>
<value>192.168.21.219</value>
<tag>OZONE, SCM, REQUIRED</tag>
<description>
The address of the Ozone SCM client service. This is a required setting.
It is a string in the host:port format. The port number is optional
and defaults to 9860.
</description>
</property>
<property>
<name>ozone.scm.names</name>
<value>192.168.21.219</value>
<tag>OZONE, REQUIRED</tag>
<description>
The value of this property is a set of DNS | DNS:PORT | IP
Address | IP:PORT. Written as a comma separated string. e.g. scm1,
scm2:8020, 7.7.7.7:7777.
This property allows datanodes to discover where SCM is, so that
datanodes can send heartbeat to SCM.
</description>
</property>
<property>
<name>hdds.datanode.dir</name>
<value>/data01/ozone/dn01,/data01/ozone/dn02,/data01/ozone/dn03</value>
<tag>OZONE, CONTAINER, STORAGE, MANAGEMENT</tag>
<description>Determines where on the local filesystem HDDS data will be
stored. Defaults to dfs.datanode.data.dir if not specified.
The directories should be tagged with corresponding storage types
([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for storage policies. The default
storage type will be DISK if the directory does not have a storage type
tagged explicitly.
</description>
</property>
</configuration>
4: ozone version
[root@node221 logs]# ozone version
//////////////
////////////////////
//////// ////////////////
////// ////////////////
///// //////////////// /
///// //////// ///
//// //////// /////
///// ////////////////
///// //////////////// //
//// /////////////// /////
///// /////////////// ////
///// ////// /////
////// ////// /////
/////////// ////////
////// ////////////
/// //////////
/ 1.0.0(Denali)
Source code repository Unknown -r Unknown
Compiled by root on 2021-04-02T09:53Z
Compiled with protoc 2.5.0
From source with checksum 49ea4ac8358b9bfbdcda09565e1fcd8
Using HDDS 1.0.0
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Re: I have a problem with Ozone 1.0.0
Posted by Sammi Chen <sa...@apache.org>.
These properties are required on both SCM and DN. And a cluster reboot is
required.
On Wed, Apr 7, 2021 at 11:44 AM dongyanhui@nnct-nsn.com <
dongyanhui@nnct-nsn.com> wrote:
> Thank you for update:)
>
> The same issue when i configed the parameters you give me.
>
>
>
> [root@node219 hadoop]# cat ozone-site.xml
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <configuration>
> <property>
> <name>ozone.om.address</name>
> <value>192.168.21.219</value>
> <tag>OM, REQUIRED</tag>
> <description>
> The address of the Ozone OM service. This allows clients to discover
> the address of the OM.
> </description>
> </property>
> <property>
> <name>ozone.metadata.dirs</name>
> <value>/data01/ozone</value>
> <tag>OZONE, OM, SCM, CONTAINER, STORAGE, REQUIRED</tag>
> <description>
>
> This setting is the fallback location for SCM, OM, Recon and DataNodes
> to store their metadata. This setting may be used only in test/PoC
> clusters to simplify configuration.
>
> For production clusters or any time you care about performance, it is
> recommended that ozone.om.db.dirs, ozone.scm.db.dirs and
> dfs.container.ratis.datanode.storage.dir be configured separately.
> </description>
> </property>
> <property>
> <name>ozone.scm.client.address</name>
> <value>192.168.21.219</value>
> <tag>OZONE, SCM, REQUIRED</tag>
> <description>
>
> The address of the Ozone SCM client service. This is a required setting.
>
> It is a string in the host:port format. The port number is optional
> and defaults to 9860.
> </description>
> </property>
> <property>
> <name>ozone.scm.names</name>
> <value>192.168.21.219</value>
> <tag>OZONE, REQUIRED</tag>
> <description>
> The value of this property is a set of DNS | DNS:PORT | IP
> Address | IP:PORT. Written as a comma separated string. e.g. scm1,
> scm2:8020, 7.7.7.7:7777.
> This property allows datanodes to discover where SCM is, so that
> datanodes can send heartbeat to SCM.
> </description>
> </property>
> <property>
> <name>hdds.datanode.dir</name>
> <value>/data01/ozone/dn01</value>
> <tag>OZONE, CONTAINER, STORAGE, MANAGEMENT</tag>
>
> <description>Determines where on the local filesystem HDDS data will be
> stored. Defaults to dfs.datanode.data.dir if not specified.
>
> The directories should be tagged with corresponding storage types
>
> ([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for storage policies. The default
>
> storage type will be DISK if the directory does not have a storage type
> tagged explicitly.
> </description>
> </property>
> <property>
> <name>ozone.scm.heartbeat.rpc-timeout</name>
> <value>30s</value>
> <tag>OZONE, MANAGEMENT</tag>
> <description>
> Timeout value for the RPC from Datanode to SCM.
> </description>
> </property>
>
> <property>
> <name>ozone.scm.heartbeat.rpc-retry-interval</name>
> <value>60s</value>
> <tag>OZONE, MANAGEMENT</tag>
> <description>
> Retry interval for the RPC from Datanode to SCM.
> Make sure rpc-retry-count * (rpc-timeout + rpc-retry-interval)
> is less than hdds.heartbeat.interval.
> </description>
> </property>
> <property>
> <name>ozone.scm.pipeline.creation.auto.factor.one</name>
> <value>true</value>
> <tag>OZONE, SCM, PIPELINE</tag>
> <description>
> If enabled, SCM will auto create RATIS factor ONE pipeline.
> </description>
> </property>
> <property>
> <name>ozone.scm.datanode.pipeline.limit</name>
> <value>5</value>
> <tag>OZONE, SCM, PIPELINE</tag>
> <description>Max number of pipelines per datanode can be engaged in.
>
> Setting the value to 0 means the pipeline limit per dn will be determined
> by the no of metadata volumes reported per dn.
> </description>
> </property>
> </configuration>
>
>
>
> ------------------------------
> 此致，敬礼！
> 董延辉
> 电话：13633860082
> Email:dongyanhui@nnct-nsn.com
>
>
> *From:* Sammi Chen <sa...@apache.org>
> *Date:* 2021-04-07 10:22
> *To:* dev <de...@ozone.apache.org>
> *CC:* dongyanhui <do...@nnct-nsn.com>
> *Subject:* Re: I have a problem with Ozone 1.0.0
> Hey dongyanhui <do...@nnct-nsn.com>，
>
> From the " ozone admin datanode list " output, we can see that there is
> only one THREE factor pipeline, which involves three datanode 219, 220,
> 225. I guess your data requires three replicas, so that your data is
> written to these three datanodes.
>
> To tuning the cluster, you can try,
> 1. controls whether factor one pipeline should be created. If you don't
> need to write one replica data, you can turn this off .
> <property>
> <name>ozone.scm.pipeline.creation.auto.factor.one</name>
> <value>true</value>
> <tag>OZONE, SCM, PIPELINE</tag>
> <description>
> If enabled, SCM will auto create RATIS factor ONE pipeline.
> </description>
> </property>
>
> 2. controls how many pipelines can a DN have. You can increase this
> number, say 5, if you have more than 5 DN disks configured.
>
> <property>
> <name>ozone.scm.datanode.pipeline.limit</name>
> <value>2</value>
> <tag>OZONE, SCM, PIPELINE</tag>
> <description>Max number of pipelines per datanode can be engaged in.
> Setting the value to 0 means the pipeline limit per dn will be
> determined
> by the no of metadata volumes reported per dn.
> </description>
> </property>
>
>
> Regards,
> Sammi
>
>
>
> On Tue, Apr 6, 2021 at 8:41 PM Elek, Marton <el...@apache.org> wrote:
>
>>
>>
>>
>> -------- Forwarded Message --------
>> Subject: I have a problem with Ozone 1.0.0
>> Date: Tue, 6 Apr 2021 12:16:57 +0800
>> From: dongyanhui@nnct-nsn.com <do...@nnct-nsn.com>
>> To: issues <is...@ozone.apache.org>
>>
>>
>>
>>
>> Hi,Support!
>>
>> 1：I have installed the Ozone 1.0.1 sucessfully with 5 datanodes.
>>
>> [root@node221 logs]# ozone admin datanode list
>> Datanode: 3e970fc5-c0a3-46c0-812f-9743ea879fce (/default-rack/
>> 192.168.21.226/node226.cluster1.com/1 pipelines)
>>
>> Related pipelines:
>> 9004918c-6026-49a4-a024-5477cef464ab/ONE/RATIS/OPEN/Leader
>>
>> Datanode: e18fd3bf-009f-4d45-a358-4a8efdd4ef30 (/default-rack/
>> 192.168.21.221/node221.cluster1.com/1 pipelines)
>>
>> Related pipelines:
>> e75d2266-2aa1-49ad-ac16-0ed6895c9550/ONE/RATIS/OPEN/Leader
>>
>> Datanode: d07c8e6b-566f-4b5d-a21c-648a053c5b45 (/default-rack/
>> 192.168.21.220/node220.cluster1.com/2 pipelines)
>>
>> Related pipelines:
>> 5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Follower
>> e000dffa-8a70-4833-9a9b-eb13e367e7dd/ONE/RATIS/OPEN/Leader
>>
>> Datanode: 951aef66-9b85-4942-a218-a25f758f192d (/default-rack/
>> 192.168.21.219/node219.cluster1.com/2 pipelines)
>>
>> Related pipelines:
>> 5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Follower
>> 54d792ec-5117-4678-951e-e22ca99bf5e5/ONE/RATIS/OPEN/Leader
>>
>> Datanode: f3e011c6-e08e-4aba-8b43-dc4bfa87f752 (/default-rack/
>> 192.168.21.225/node225.cluster1.com/2 pipelines)
>>
>> Related pipelines:
>> 3f6d26d2-52c2-41f9-8e8f-ede7c111074f/ONE/RATIS/OPEN/Leader
>> 5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Leader
>>
>> 2：I have a problem with Ozone 1.0.0 as bellow:
>>
>> The dns can increase more than 3 nodes, but The data can upload to 3
>> datanodes only,*is their licensing restrictions?*
>>
>>
>> [root@node219 hadoop]# du -sh /data01/ozone/*
>> 4.0K /data01/ozone/datanode.id
>> 4.0K /data01/ozone/db.checkpoints
>> 7.4G /data01/ozone/dn01
>> 7.2G /data01/ozone/dn02
>> 7.7G /data01/ozone/dn03
>> 12K /data01/ozone/om
>> 146M /data01/ozone/om.db
>> 4.0K /data01/ozone/omMetrics
>> 6.6M /data01/ozone/ratis
>> 12K /data01/ozone/scm
>> 146M /data01/ozone/scm.db
>>
>> [root@node220 ~]# du -sh /data01/ozone/*
>> 4.0K /data01/ozone/datanode.id
>> 7.5G /data01/ozone/dn01
>> 7.3G /data01/ozone/dn02
>> 7.7G /data01/ozone/dn03
>> 6.7M /data01/ozone/ratis
>>
>> [root@node221 logs]# du -sh /data01/ozone/*
>> 4.0K /data01/ozone/datanode.id
>> 20K /data01/ozone/dn01
>> 20K /data01/ozone/dn02
>> 20K /data01/ozone/dn03
>> 48K /data01/ozone/ratis
>>
>>
>> [root@node225 ~]# du -sh /data01/ozone/*
>> 4.0K /data01/ozone/datanode.id
>> 7.7G /data01/ozone/dn01
>> 7.3G /data01/ozone/dn02
>> 7.9G /data01/ozone/dn03
>> 6.7M /data01/ozone/ratis
>>
>> [root@node226 ~]# du -sh /data01/ozone/*
>> 4.0K /data01/ozone/datanode.id
>> 16K /data01/ozone/dn01
>> 16K /data01/ozone/dn02
>> 16K /data01/ozone/dn03
>> 44K /data01/ozone/ratis
>>
>>
>> 3: The ozone config as bellow:
>>
>> [root@node219 hadoop]# cat ozone-site.xml
>> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
>> <configuration>
>> <property>
>> <name>ozone.om.address</name>
>> <value>192.168.21.219</value>
>> <tag>OM, REQUIRED</tag>
>> <description>
>>
>> The address of the Ozone OM service. This allows clients to discover
>> the address of the OM.
>> </description>
>> </property>
>> <property>
>> <name>ozone.metadata.dirs</name>
>> <value>/data01/ozone</value>
>> <tag>OZONE, OM, SCM, CONTAINER, STORAGE, REQUIRED</tag>
>> <description>
>>
>>
>> This setting is the fallback location for SCM, OM, Recon and DataNodes
>> to store their metadata. This setting may be used only in test/PoC
>> clusters to simplify configuration.
>>
>>
>>
>> For production clusters or any time you care about performance, it is
>> recommended that ozone.om.db.dirs, ozone.scm.db.dirs and
>> dfs.container.ratis.datanode.storage.dir be configured separately.
>> </description>
>> </property>
>> <property>
>> <name>ozone.scm.client.address</name>
>> <value>192.168.21.219</value>
>> <tag>OZONE, SCM, REQUIRED</tag>
>> <description>
>>
>>
>> The address of the Ozone SCM client service. This is a required setting.
>>
>>
>> It is a string in the host:port format. The port number is optional
>> and defaults to 9860.
>> </description>
>> </property>
>> <property>
>> <name>ozone.scm.names</name>
>> <value>192.168.21.219</value>
>> <tag>OZONE, REQUIRED</tag>
>> <description>
>> The value of this property is a set of DNS | DNS:PORT | IP
>> Address | IP:PORT. Written as a comma separated string. e.g. scm1,
>> scm2:8020, 7.7.7.7:7777.
>> This property allows datanodes to discover where SCM is, so that
>> datanodes can send heartbeat to SCM.
>> </description>
>> </property>
>> <property>
>> <name>hdds.datanode.dir</name>
>>
>>
>> <value>/data01/ozone/dn01,/data01/ozone/dn02,/data01/ozone/dn03</value>
>> <tag>OZONE, CONTAINER, STORAGE, MANAGEMENT</tag>
>>
>>
>> <description>Determines where on the local filesystem HDDS data will be
>>
>>
>> stored. Defaults to dfs.datanode.data.dir if not specified.
>>
>>
>> The directories should be tagged with corresponding storage types
>>
>>
>> ([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for storage policies. The default
>>
>>
>> storage type will be DISK if the directory does not have a storage type
>> tagged explicitly.
>> </description>
>> </property>
>> </configuration>
>>
>> 4: ozone version
>>
>>
>> [root@node221 logs]# ozone version
>> //////////////
>> ////////////////////
>> //////// ////////////////
>> ////// ////////////////
>> ///// //////////////// /
>> ///// //////// ///
>> //// //////// /////
>> ///// ////////////////
>> ///// //////////////// //
>> //// /////////////// /////
>> ///// /////////////// ////
>> ///// ////// /////
>> ////// ////// /////
>> /////////// ////////
>> ////// ////////////
>> /// //////////
>> / 1.0.0(Denali)
>>
>> Source code repository Unknown -r Unknown
>> Compiled by root on 2021-04-02T09:53Z
>> Compiled with protoc 2.5.0
>> From source with checksum 49ea4ac8358b9bfbdcda09565e1fcd8
>>
>> Using HDDS 1.0.0
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
>> For additional commands, e-mail: dev-help@ozone.apache.org
>>
>>
Re: I have a problem with Ozone 1.0.0
Posted by "Elek, Marton" <el...@apache.org>.
On 4/7/21 5:43 AM, dongyanhui@nnct-nsn.com wrote:
> Thank you for update:)
>
> The same issue when i configed the parameters you give me.
Hi,
How does the same issue look like with the new configuration?
1. Do you see new pipelines created with `ozone admin pipeline list`
2. What is the data distribution after the update config?
3. Did you write enough data to use multiple containers (one container
is 5g. If you upload data to the same container, the same 3-node
pipeline will be used...)
Marton
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: Re: I have a problem with Ozone 1.0.0
Posted by "dongyanhui@nnct-nsn.com" <do...@nnct-nsn.com>.
Thank you for update:)
The same issue when i configed the parameters you give me.
[root@node219 hadoop]# cat ozone-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
<property>
<name>ozone.om.address</name>
<value>192.168.21.219</value>
<tag>OM, REQUIRED</tag>
<description>
The address of the Ozone OM service. This allows clients to discover
the address of the OM.
</description>
</property>
<property>
<name>ozone.metadata.dirs</name>
<value>/data01/ozone</value>
<tag>OZONE, OM, SCM, CONTAINER, STORAGE, REQUIRED</tag>
<description>
This setting is the fallback location for SCM, OM, Recon and DataNodes
to store their metadata. This setting may be used only in test/PoC
clusters to simplify configuration.
For production clusters or any time you care about performance, it is
recommended that ozone.om.db.dirs, ozone.scm.db.dirs and
dfs.container.ratis.datanode.storage.dir be configured separately.
</description>
</property>
<property>
<name>ozone.scm.client.address</name>
<value>192.168.21.219</value>
<tag>OZONE, SCM, REQUIRED</tag>
<description>
The address of the Ozone SCM client service. This is a required setting.
It is a string in the host:port format. The port number is optional
and defaults to 9860.
</description>
</property>
<property>
<name>ozone.scm.names</name>
<value>192.168.21.219</value>
<tag>OZONE, REQUIRED</tag>
<description>
The value of this property is a set of DNS | DNS:PORT | IP
Address | IP:PORT. Written as a comma separated string. e.g. scm1,
scm2:8020, 7.7.7.7:7777.
This property allows datanodes to discover where SCM is, so that
datanodes can send heartbeat to SCM.
</description>
</property>
<property>
<name>hdds.datanode.dir</name>
<value>/data01/ozone/dn01</value>
<tag>OZONE, CONTAINER, STORAGE, MANAGEMENT</tag>
<description>Determines where on the local filesystem HDDS data will be
stored. Defaults to dfs.datanode.data.dir if not specified.
The directories should be tagged with corresponding storage types
([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for storage policies. The default
storage type will be DISK if the directory does not have a storage type
tagged explicitly.
</description>
</property>
<property>
<name>ozone.scm.heartbeat.rpc-timeout</name>
<value>30s</value>
<tag>OZONE, MANAGEMENT</tag>
<description>
Timeout value for the RPC from Datanode to SCM.
</description>
</property>
<property>
<name>ozone.scm.heartbeat.rpc-retry-interval</name>
<value>60s</value>
<tag>OZONE, MANAGEMENT</tag>
<description>
Retry interval for the RPC from Datanode to SCM.
Make sure rpc-retry-count * (rpc-timeout + rpc-retry-interval)
is less than hdds.heartbeat.interval.
</description>
</property>
<property>
<name>ozone.scm.pipeline.creation.auto.factor.one</name>
<value>true</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>
If enabled, SCM will auto create RATIS factor ONE pipeline.
</description>
</property>
<property>
<name>ozone.scm.datanode.pipeline.limit</name>
<value>5</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>Max number of pipelines per datanode can be engaged in.
Setting the value to 0 means the pipeline limit per dn will be determined
by the no of metadata volumes reported per dn.
</description>
</property>
</configuration>
此致，敬礼！
董延辉
电话：13633860082
Email:dongyanhui@nnct-nsn.com
From: Sammi Chen
Date: 2021-04-07 10:22
To: dev
CC: dongyanhui
Subject: Re: I have a problem with Ozone 1.0.0
Hey dongyanhui，
From the " ozone admin datanode list " output, we can see that there is only one THREE factor pipeline, which involves three datanode 219, 220, 225. I guess your data requires three replicas, so that your data is written to these three datanodes.
To tuning the cluster, you can try,
1. controls whether factor one pipeline should be created. If you don't need to write one replica data, you can turn this off .
<property>
<name>ozone.scm.pipeline.creation.auto.factor.one</name>
<value>true</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>
If enabled, SCM will auto create RATIS factor ONE pipeline.
</description>
</property>
2. controls how many pipelines can a DN have. You can increase this number, say 5, if you have more than 5 DN disks configured.
<property>
<name>ozone.scm.datanode.pipeline.limit</name>
<value>2</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>Max number of pipelines per datanode can be engaged in.
Setting the value to 0 means the pipeline limit per dn will be determined
by the no of metadata volumes reported per dn.
</description>
</property>
Regards,
Sammi
On Tue, Apr 6, 2021 at 8:41 PM Elek, Marton <el...@apache.org> wrote:
-------- Forwarded Message --------
Subject: I have a problem with Ozone 1.0.0
Date: Tue, 6 Apr 2021 12:16:57 +0800
From: dongyanhui@nnct-nsn.com <do...@nnct-nsn.com>
To: issues <is...@ozone.apache.org>
Hi,Support!
1：I have installed the Ozone 1.0.1 sucessfully with 5 datanodes.
[root@node221 logs]# ozone admin datanode list
Datanode: 3e970fc5-c0a3-46c0-812f-9743ea879fce (/default-rack/192.168.21.226/node226.cluster1.com/1 pipelines)
Related pipelines:
9004918c-6026-49a4-a024-5477cef464ab/ONE/RATIS/OPEN/Leader
Datanode: e18fd3bf-009f-4d45-a358-4a8efdd4ef30 (/default-rack/192.168.21.221/node221.cluster1.com/1 pipelines)
Related pipelines:
e75d2266-2aa1-49ad-ac16-0ed6895c9550/ONE/RATIS/OPEN/Leader
Datanode: d07c8e6b-566f-4b5d-a21c-648a053c5b45 (/default-rack/192.168.21.220/node220.cluster1.com/2 pipelines)
Related pipelines:
5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Follower
e000dffa-8a70-4833-9a9b-eb13e367e7dd/ONE/RATIS/OPEN/Leader
Datanode: 951aef66-9b85-4942-a218-a25f758f192d (/default-rack/192.168.21.219/node219.cluster1.com/2 pipelines)
Related pipelines:
5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Follower
54d792ec-5117-4678-951e-e22ca99bf5e5/ONE/RATIS/OPEN/Leader
Datanode: f3e011c6-e08e-4aba-8b43-dc4bfa87f752 (/default-rack/192.168.21.225/node225.cluster1.com/2 pipelines)
Related pipelines:
3f6d26d2-52c2-41f9-8e8f-ede7c111074f/ONE/RATIS/OPEN/Leader
5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Leader
2：I have a problem with Ozone 1.0.0 as bellow:
The dns can increase more than 3 nodes, but The data can upload to 3
datanodes only,*is their licensing restrictions?*
[root@node219 hadoop]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
4.0K /data01/ozone/db.checkpoints
7.4G /data01/ozone/dn01
7.2G /data01/ozone/dn02
7.7G /data01/ozone/dn03
12K /data01/ozone/om
146M /data01/ozone/om.db
4.0K /data01/ozone/omMetrics
6.6M /data01/ozone/ratis
12K /data01/ozone/scm
146M /data01/ozone/scm.db
[root@node220 ~]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
7.5G /data01/ozone/dn01
7.3G /data01/ozone/dn02
7.7G /data01/ozone/dn03
6.7M /data01/ozone/ratis
[root@node221 logs]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
20K /data01/ozone/dn01
20K /data01/ozone/dn02
20K /data01/ozone/dn03
48K /data01/ozone/ratis
[root@node225 ~]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
7.7G /data01/ozone/dn01
7.3G /data01/ozone/dn02
7.9G /data01/ozone/dn03
6.7M /data01/ozone/ratis
[root@node226 ~]# du -sh /data01/ozone/*
4.0K /data01/ozone/datanode.id
16K /data01/ozone/dn01
16K /data01/ozone/dn02
16K /data01/ozone/dn03
44K /data01/ozone/ratis
3: The ozone config as bellow:
[root@node219 hadoop]# cat ozone-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
<property>
<name>ozone.om.address</name>
<value>192.168.21.219</value>
<tag>OM, REQUIRED</tag>
<description>
The address of the Ozone OM service. This allows clients to discover
the address of the OM.
</description>
</property>
<property>
<name>ozone.metadata.dirs</name>
<value>/data01/ozone</value>
<tag>OZONE, OM, SCM, CONTAINER, STORAGE, REQUIRED</tag>
<description>
This setting is the fallback location for SCM, OM, Recon and DataNodes
to store their metadata. This setting may be used only in test/PoC
clusters to simplify configuration.
For production clusters or any time you care about performance, it is
recommended that ozone.om.db.dirs, ozone.scm.db.dirs and
dfs.container.ratis.datanode.storage.dir be configured separately.
</description>
</property>
<property>
<name>ozone.scm.client.address</name>
<value>192.168.21.219</value>
<tag>OZONE, SCM, REQUIRED</tag>
<description>
The address of the Ozone SCM client service. This is a required setting.
It is a string in the host:port format. The port number is optional
and defaults to 9860.
</description>
</property>
<property>
<name>ozone.scm.names</name>
<value>192.168.21.219</value>
<tag>OZONE, REQUIRED</tag>
<description>
The value of this property is a set of DNS | DNS:PORT | IP
Address | IP:PORT. Written as a comma separated string. e.g. scm1,
scm2:8020, 7.7.7.7:7777.
This property allows datanodes to discover where SCM is, so that
datanodes can send heartbeat to SCM.
</description>
</property>
<property>
<name>hdds.datanode.dir</name>
<value>/data01/ozone/dn01,/data01/ozone/dn02,/data01/ozone/dn03</value>
<tag>OZONE, CONTAINER, STORAGE, MANAGEMENT</tag>
<description>Determines where on the local filesystem HDDS data will be
stored. Defaults to dfs.datanode.data.dir if not specified.
The directories should be tagged with corresponding storage types
([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for storage policies. The default
storage type will be DISK if the directory does not have a storage type
tagged explicitly.
</description>
</property>
</configuration>
4: ozone version
[root@node221 logs]# ozone version
//////////////
////////////////////
//////// ////////////////
////// ////////////////
///// //////////////// /
///// //////// ///
//// //////// /////
///// ////////////////
///// //////////////// //
//// /////////////// /////
///// /////////////// ////
///// ////// /////
////// ////// /////
/////////// ////////
////// ////////////
/// //////////
/ 1.0.0(Denali)
Source code repository Unknown -r Unknown
Compiled by root on 2021-04-02T09:53Z
Compiled with protoc 2.5.0
From source with checksum 49ea4ac8358b9bfbdcda09565e1fcd8
Using HDDS 1.0.0
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
For additional commands, e-mail: dev-help@ozone.apache.org
Re: I have a problem with Ozone 1.0.0
Posted by Sammi Chen <sa...@apache.org>.
Hey dongyanhui <do...@nnct-nsn.com>，
From the " ozone admin datanode list " output, we can see that there is
only one THREE factor pipeline, which involves three datanode 219, 220,
225. I guess your data requires three replicas, so that your data is
written to these three datanodes.
To tuning the cluster, you can try,
1. controls whether factor one pipeline should be created. If you don't
need to write one replica data, you can turn this off .
<property>
<name>ozone.scm.pipeline.creation.auto.factor.one</name>
<value>true</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>
If enabled, SCM will auto create RATIS factor ONE pipeline.
</description>
</property>
2. controls how many pipelines can a DN have. You can increase this
number, say 5, if you have more than 5 DN disks configured.
<property>
<name>ozone.scm.datanode.pipeline.limit</name>
<value>2</value>
<tag>OZONE, SCM, PIPELINE</tag>
<description>Max number of pipelines per datanode can be engaged in.
Setting the value to 0 means the pipeline limit per dn will be
determined
by the no of metadata volumes reported per dn.
</description>
</property>
Regards,
Sammi
On Tue, Apr 6, 2021 at 8:41 PM Elek, Marton <el...@apache.org> wrote:
>
>
>
> -------- Forwarded Message --------
> Subject: I have a problem with Ozone 1.0.0
> Date: Tue, 6 Apr 2021 12:16:57 +0800
> From: dongyanhui@nnct-nsn.com <do...@nnct-nsn.com>
> To: issues <is...@ozone.apache.org>
>
>
>
>
> Hi,Support!
>
> 1：I have installed the Ozone 1.0.1 sucessfully with 5 datanodes.
>
> [root@node221 logs]# ozone admin datanode list
> Datanode: 3e970fc5-c0a3-46c0-812f-9743ea879fce (/default-rack/
> 192.168.21.226/node226.cluster1.com/1 pipelines)
>
> Related pipelines:
> 9004918c-6026-49a4-a024-5477cef464ab/ONE/RATIS/OPEN/Leader
>
> Datanode: e18fd3bf-009f-4d45-a358-4a8efdd4ef30 (/default-rack/
> 192.168.21.221/node221.cluster1.com/1 pipelines)
>
> Related pipelines:
> e75d2266-2aa1-49ad-ac16-0ed6895c9550/ONE/RATIS/OPEN/Leader
>
> Datanode: d07c8e6b-566f-4b5d-a21c-648a053c5b45 (/default-rack/
> 192.168.21.220/node220.cluster1.com/2 pipelines)
>
> Related pipelines:
> 5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Follower
> e000dffa-8a70-4833-9a9b-eb13e367e7dd/ONE/RATIS/OPEN/Leader
>
> Datanode: 951aef66-9b85-4942-a218-a25f758f192d (/default-rack/
> 192.168.21.219/node219.cluster1.com/2 pipelines)
>
> Related pipelines:
> 5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Follower
> 54d792ec-5117-4678-951e-e22ca99bf5e5/ONE/RATIS/OPEN/Leader
>
> Datanode: f3e011c6-e08e-4aba-8b43-dc4bfa87f752 (/default-rack/
> 192.168.21.225/node225.cluster1.com/2 pipelines)
>
> Related pipelines:
> 3f6d26d2-52c2-41f9-8e8f-ede7c111074f/ONE/RATIS/OPEN/Leader
> 5e8f70e1-0939-4ff7-9da7-8d65524c330d/THREE/RATIS/OPEN/Leader
>
> 2：I have a problem with Ozone 1.0.0 as bellow:
>
> The dns can increase more than 3 nodes, but The data can upload to 3
> datanodes only,*is their licensing restrictions?*
>
>
> [root@node219 hadoop]# du -sh /data01/ozone/*
> 4.0K /data01/ozone/datanode.id
> 4.0K /data01/ozone/db.checkpoints
> 7.4G /data01/ozone/dn01
> 7.2G /data01/ozone/dn02
> 7.7G /data01/ozone/dn03
> 12K /data01/ozone/om
> 146M /data01/ozone/om.db
> 4.0K /data01/ozone/omMetrics
> 6.6M /data01/ozone/ratis
> 12K /data01/ozone/scm
> 146M /data01/ozone/scm.db
>
> [root@node220 ~]# du -sh /data01/ozone/*
> 4.0K /data01/ozone/datanode.id
> 7.5G /data01/ozone/dn01
> 7.3G /data01/ozone/dn02
> 7.7G /data01/ozone/dn03
> 6.7M /data01/ozone/ratis
>
> [root@node221 logs]# du -sh /data01/ozone/*
> 4.0K /data01/ozone/datanode.id
> 20K /data01/ozone/dn01
> 20K /data01/ozone/dn02
> 20K /data01/ozone/dn03
> 48K /data01/ozone/ratis
>
>
> [root@node225 ~]# du -sh /data01/ozone/*
> 4.0K /data01/ozone/datanode.id
> 7.7G /data01/ozone/dn01
> 7.3G /data01/ozone/dn02
> 7.9G /data01/ozone/dn03
> 6.7M /data01/ozone/ratis
>
> [root@node226 ~]# du -sh /data01/ozone/*
> 4.0K /data01/ozone/datanode.id
> 16K /data01/ozone/dn01
> 16K /data01/ozone/dn02
> 16K /data01/ozone/dn03
> 44K /data01/ozone/ratis
>
>
> 3: The ozone config as bellow:
>
> [root@node219 hadoop]# cat ozone-site.xml
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <configuration>
> <property>
> <name>ozone.om.address</name>
> <value>192.168.21.219</value>
> <tag>OM, REQUIRED</tag>
> <description>
>
> The address of the Ozone OM service. This allows clients to discover
> the address of the OM.
> </description>
> </property>
> <property>
> <name>ozone.metadata.dirs</name>
> <value>/data01/ozone</value>
> <tag>OZONE, OM, SCM, CONTAINER, STORAGE, REQUIRED</tag>
> <description>
>
>
> This setting is the fallback location for SCM, OM, Recon and DataNodes
> to store their metadata. This setting may be used only in test/PoC
> clusters to simplify configuration.
>
>
>
> For production clusters or any time you care about performance, it is
> recommended that ozone.om.db.dirs, ozone.scm.db.dirs and
> dfs.container.ratis.datanode.storage.dir be configured separately.
> </description>
> </property>
> <property>
> <name>ozone.scm.client.address</name>
> <value>192.168.21.219</value>
> <tag>OZONE, SCM, REQUIRED</tag>
> <description>
>
>
> The address of the Ozone SCM client service. This is a required setting.
>
> It is a string in the host:port format. The port number is optional
> and defaults to 9860.
> </description>
> </property>
> <property>
> <name>ozone.scm.names</name>
> <value>192.168.21.219</value>
> <tag>OZONE, REQUIRED</tag>
> <description>
> The value of this property is a set of DNS | DNS:PORT | IP
> Address | IP:PORT. Written as a comma separated string. e.g. scm1,
> scm2:8020, 7.7.7.7:7777.
> This property allows datanodes to discover where SCM is, so that
> datanodes can send heartbeat to SCM.
> </description>
> </property>
> <property>
> <name>hdds.datanode.dir</name>
>
>
> <value>/data01/ozone/dn01,/data01/ozone/dn02,/data01/ozone/dn03</value>
> <tag>OZONE, CONTAINER, STORAGE, MANAGEMENT</tag>
>
>
> <description>Determines where on the local filesystem HDDS data will be
>
>
> stored. Defaults to dfs.datanode.data.dir if not specified.
>
>
> The directories should be tagged with corresponding storage types
>
>
> ([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for storage policies. The default
>
>
> storage type will be DISK if the directory does not have a storage type
> tagged explicitly.
> </description>
> </property>
> </configuration>
>
> 4: ozone version
>
>
> [root@node221 logs]# ozone version
> //////////////
> ////////////////////
> //////// ////////////////
> ////// ////////////////
> ///// //////////////// /
> ///// //////// ///
> //// //////// /////
> ///// ////////////////
> ///// //////////////// //
> //// /////////////// /////
> ///// /////////////// ////
> ///// ////// /////
> ////// ////// /////
> /////////// ////////
> ////// ////////////
> /// //////////
> / 1.0.0(Denali)
>
> Source code repository Unknown -r Unknown
> Compiled by root on 2021-04-02T09:53Z
> Compiled with protoc 2.5.0
> From source with checksum 49ea4ac8358b9bfbdcda09565e1fcd8
>
> Using HDDS 1.0.0
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ozone.apache.org
> For additional commands, e-mail: dev-help@ozone.apache.org
>
>