You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Damien Hardy <dh...@figarocms.fr> on 2011/09/08 18:18:11 UTC
Where are "Non DFS,Used" ?
Hello there,
My Web interface DFS Home announce me that I have 55.52 GB of non DFS
used data on my 2 nodes cluster.
I would like to know what are those data about.
Live Datanodes : 2
Node Last
Contact Admin State Configured
Capacity (GB) Used
(GB) Non DFS
Used (GB) Remaining
(GB) Used
(%) Used
(%) Remaining
(%) Blocks Failed Volumes
hadoop001
<http://hadoop001.back.adencf.local:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F>
2 In Service 336.41 157.72 27.87 150.82 46.88
44.83 6012 0
hadoop002
<http://hadoop002.back.adencf.local:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F>
2 In Service 336.41 157.72 27.65 151.04 46.88
44.9 6162 0
------------------------------------------------------------------------
root@hadoop001:/etc/hadoop/conf# df -hP /var/ /tmp
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg00-var 337G 169G 151G 53% /var
/dev/mapper/vg00-tmp 473M 13M 436M 3% /tmp
root@hadoop001:/etc/hadoop/conf# du -sh /var/*
4.0K /var/agentx
5.4M /var/backups
643M /var/cache
158G /var/lib
4.0K /var/local
0 /var/lock
9.8G /var/log
16K /var/lost+found
4.0K /var/mail
4.0K /var/opt
124K /var/run
676K /var/spool
60K /var/tmp
21M /var/zookeeper
root@hadoop001:/etc/hadoop/conf# du -sh
/var/lib/hadoop-0.20/datanode/hdfs/dfs/data/
159G /var/lib/hadoop-0.20/datanode/hdfs/dfs/data/
root@hadoop002:/etc/hadoop/conf# df -hP /var/ /tmp
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg00-var 337G 169G 151G 53% /var
/dev/mapper/vg00-tmp 473M 11M 438M 3% /tmp
root@hadoop002:/etc/hadoop/conf# du -sh /var/*
4.0K /var/agentx
5.3M /var/backups
102M /var/cache
158G /var/lib
4.0K /var/local
0 /var/lock
11G /var/log
16K /var/lost+found
4.0K /var/mail
4.0K /var/opt
104K /var/run
652K /var/spool
32K /var/tmp
21M /var/zookeeper
root@hadoop002:/etc/hadoop/conf# du -sh
/var/lib/hadoop-0.20/datanode/hdfs/dfs/data/
159G /var/lib/hadoop-0.20/datanode/hdfs/dfs/data/
All the things in /var/lib/hadoop-0.20/datanode/hdfs/dfs/data/ was
create by the cluster datanodes.
My question is where are hidden those 27Go of "non dfs data" on each
node (I can find 15Go at max on each which are mainly in /var/logs).
Thanks for your helps.
Regards,
--
Damien