You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by 林家銘 <ro...@gmail.com> on 2015/10/14 02:23:12 UTC

Datanode install on cluster Dockers

Hi

I install three datanodes ( Hadoop 2.7.1) in a multiple Docker cluster
which is connected by Flannel.
The bridge on each physical node are

physical host 1:
IP address for flannel0: 172.16.91.0
IP address for docker0:  172.16.91.1

physical host 2:
IP address for flannel0: 172.16.46.0
IP address for docker0:  172.16.46.1

physical host 3:
IP address for flannel0: 172.16.31.0
IP address for docker0:  172.16.31.1

There are two datanodes are installed on "physical host 1", with IPs
172.16.91.19 and 172.16.91.20
One on "physical host 2" with IP 172.16.46.22
Namenode is installed on "physical host 2" with IP 172.16.46.21

And the problem is when I check the HDFS health status there is only
two datanodes appear.

[hdfs@amb1-host-dev bin]$ hadoop dfsadmin -report
Name: 172.16.46.22:50010(k8s_POD-afaa08c2_amb4-rc-dev-n35n9_default_42432432-71d9-11e5-8d52-000c2900be4e_5fee9eed)
Hostname: amb4-host-dev
Decommission Status : Normal
Configured Capacity: 87571161088 (81.56 GB)
DFS Used: 33554432 (32 MB)
Non DFS Used: 32210010112 (30.00 GB)
DFS Remaining: 55327596544 (51.53 GB)
DFS Used%: 0.04%
DFS Remaining%: 63.18%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 2
Last contact: Wed Oct 14 00:16:42 UTC 2015


Name: 172.16.91.0:50010 (172.16.91.0)   <-----------------------here
is the problem
Hostname: amb3-host-dev
Decommission Status : Normal
Configured Capacity: 87571161088 (81.56 GB)
DFS Used: 28672 (28 KB)
Non DFS Used: 36738072576 (34.21 GB)
DFS Remaining: 50833059840 (47.34 GB)
DFS Used%: 0.00%
DFS Remaining%: 58.05%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 2
Last contact: Wed Oct 14 00:16:41 UTC 2015

Why there is only one datanode on "physical host2", and why HDFS
regards the whole physical host as a datanode?