You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Manoj Govindassamy (JIRA)" <ji...@apache.org> on 2016/10/26 00:31:58 UTC
[jira] [Commented] (HDFS-11058) Implement 'hadoop fs -df' command
for ViewFileSystem
[ https://issues.apache.org/jira/browse/HDFS-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15606922#comment-15606922 ]
Manoj Govindassamy commented on HDFS-11058:
-------------------------------------------
Current {{df}} command behavior for *DFS*
{noformat}
# hadoop fs -fs hdfs://127.0.0.1:50001/ -df -h /
Filesystem Size Used Available Use%
hdfs://127.0.0.1:50001 930.6 G 96 K 652.6 G 0%
{noformat}
Current {{df}} command behavior for *ViewFileSystem*
{noformat}
# hadoop fs -fs viewfs://ClusterX/ -df -h /
Filesystem Size Used Available Use%
viewfs://ClusterX/ 8.0 E 0 8.0 E 0%
{noformat}
Just like {{df}} in traditional linux systems, here is my proposal for {{df}} command when run against *ViewFileSystem*
{noformat}
# hadoop fs -fs viewfs://ClusterX/ -df -h /
Filesystem Mounted on Size Used Available Use%
hdfs://NN0.example.com:port/ /nn0 100.0 G 0 80.0 G 0%
hdfs://NN1.example.com:port/ /nn1 100.0 G 0 80.0 G 0%
hdfs://NN2.example.com:port/ /nn2 100.0 G 0 80.0 G 0%
hdfs://NN3.example.com:port/ /nn3 100.0 G 0 80.0 G 0%
{noformat}
[~andrew.wang], [~eddyxu], [~zhz], any thoughts on the above {{df}} proposal for {{viewfs://}} ?
> Implement 'hadoop fs -df' command for ViewFileSystem
> -------------------------------------------------------
>
> Key: HDFS-11058
> URL: https://issues.apache.org/jira/browse/HDFS-11058
> Project: Hadoop HDFS
> Issue Type: Task
> Affects Versions: 3.0.0-alpha1
> Reporter: Manoj Govindassamy
> Assignee: Manoj Govindassamy
>
> Df command doesn't seem to work well with ViewFileSystem. It always reports used data as 0. Here is the client mount table configuration I am using against a federated clusters of 2 NameNodes and 2 DataNoes.
> {code}
> 1 <?xml version="1.0" ?>
> 2 <configuration>
> 3 <property>
> 4 <name>fs.defaultFS</name>
> 5 <value>viewfs://ClusterX/</value>
> 6 </property>
> ..
> 11 <property>
> 12 <name>fs.default.name</name>
> 13 <value>viewfs://ClusterX/</value>
> 14 </property>
> ..
> 23 <property>
> 24 <name>fs.viewfs.mounttable.ClusterX.link./nn0</name>
> 25 <value>hdfs://127.0.0.1:50001/</value>
> 26 </property>
> 27 <property>
> 28 <name>fs.viewfs.mounttable.ClusterX.link./nn1</name>
> 29 <value>hdfs://127.0.0.1:51001/</value>
> 30 </property>
> 31 <property>
> 32 <name>fs.viewfs.mounttable.ClusterX.link./nn2</name>
> 33 <value>hdfs://127.0.0.1:52001/nn2</value>
> 34 </property>
> 35 <property>
> 36 <name>fs.viewfs.mounttable.ClusterX.link./nn3</name>
> 37 <value>hdfs://127.0.0.1:52001/nn3</value>
> 38 </property>
> 39 <property>
> 40 <name>fs.viewfs.mounttable.ClusterY.linkMergeSlash</name>
> 41 <value>hdfs://127.0.0.1:50001/</value>
> 42 </property>
> 43 </configuration>
> {code}
> {{Df}} command always reports Size/Available as 8.0E and the usage as 0 for any federated cluster.
> {noformat}
> # hadoop fs -fs viewfs://ClusterX/ -df /
> Filesystem Size Used Available Use%
> viewfs://ClusterX/ 9223372036854775807 0 9223372036854775807 0%
> # hadoop fs -fs viewfs://ClusterX/ -df -h /
> Filesystem Size Used Available Use%
> viewfs://ClusterX/ 8.0 E 0 8.0 E 0%
> # hadoop fs -fs viewfs://ClusterY/ -df -h /
> Filesystem Size Used Available Use%
> viewfs://ClusterY/ 8.0 E 0 8.0 E 0%
> {noformat}
> Whereas {{Du}} command seems to work as expected even with ViewFileSystem.
> {noformat}
> # hadoop fs -fs viewfs://ClusterY/ -du -h /
> 10.6 K 31.8 K /build.log.16y
> 0 0 /user
> # hadoop fs -fs viewfs://ClusterX/ -du -h /
> 10.6 K 31.8 K /nn0
> 0 0 /nn1
> 20.2 K 35.8 K /nn3
> 40.6 K 34.3 K /nn4
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org