You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Pete Wyckoff (JIRA)" <ji...@apache.org> on 2008/11/18 22:33:44 UTC
[jira] Created: (HADOOP-4680) fuse-dfs - df -kh on hdfs mount shows
much less %used than the dfs UI
fuse-dfs - df -kh on hdfs mount shows much less %used than the dfs UI
---------------------------------------------------------------------
Key: HADOOP-4680
URL: https://issues.apache.org/jira/browse/HADOOP-4680
Project: Hadoop Core
Issue Type: Bug
Components: contrib/fuse-dfs
Reporter: Pete Wyckoff
i.e., statfs broken.
This used to show the correct amount and now is showing wrong numbers in production environment.
Yet, the unit test for this passes??
This is the trunk version of fuse-dfs against 0.17 libhdfs and hadoop.
Unknown if this affects 0.18.2 or 19.1 or trunk but i suspect it does.
admittedly fuse-dfs isn't supported against hadoop 17 but I am confident that it is likely broken in all versions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4680) fuse-dfs - df -kh on hdfs mount
shows much less %used than the dfs UI
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12648894#action_12648894 ]
Pete Wyckoff commented on HADOOP-4680:
--------------------------------------
This is the fix but I still haven't looked at why the unit test didn't catch it.
{code}
- st->f_bavail = cap/bsize;
+ st->f_bavail = (cap-used)/bsize;
{code}
I used this to debug:
{code}
#include <sys/statvfs.h>
#include <stdio.h>
int main(int ac, char ** av) {
struct statvfs b;
statvfs(av[1], &b);
printf("%ld\n",b.f_bsize);
printf("%ld\n",b.f_frsize);
printf("%ld\n",b.f_blocks);
printf("%ld\n",b.f_bfree);
printf("%ld\n",b.f_bavail);
printf("%ld\n",b.f_files);
printf("%ld\n",b.f_ffree);
printf("%ld\n",b.f_favail);
printf("%ld\n",b.f_fsid);
printf("%ld\n",b.f_flag);
printf("%ld\n",b.f_namemax);
}
{code}
> fuse-dfs - df -kh on hdfs mount shows much less %used than the dfs UI
> ---------------------------------------------------------------------
>
> Key: HADOOP-4680
> URL: https://issues.apache.org/jira/browse/HADOOP-4680
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fuse-dfs
> Reporter: Pete Wyckoff
>
> i.e., statfs broken.
> This used to show the correct amount and now is showing wrong numbers in production environment.
> Yet, the unit test for this passes??
> This is the trunk version of fuse-dfs against 0.17 libhdfs and hadoop.
> Unknown if this affects 0.18.2 or 19.1 or trunk but i suspect it does.
> admittedly fuse-dfs isn't supported against hadoop 17 but I am confident that it is likely broken in all versions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4680) fuse-dfs - df -kh on hdfs mount shows
much less %used than the dfs UI
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pete Wyckoff updated HADOOP-4680:
---------------------------------
Affects Version/s: 0.19.0
Fix Version/s: 0.19.1
Assignee: Pete Wyckoff
> fuse-dfs - df -kh on hdfs mount shows much less %used than the dfs UI
> ---------------------------------------------------------------------
>
> Key: HADOOP-4680
> URL: https://issues.apache.org/jira/browse/HADOOP-4680
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fuse-dfs
> Affects Versions: 0.19.0
> Reporter: Pete Wyckoff
> Assignee: Pete Wyckoff
> Fix For: 0.19.1
>
>
> i.e., statfs broken.
> This used to show the correct amount and now is showing wrong numbers in production environment.
> Yet, the unit test for this passes??
> This is the trunk version of fuse-dfs against 0.17 libhdfs and hadoop.
> Unknown if this affects 0.18.2 or 19.1 or trunk but i suspect it does.
> admittedly fuse-dfs isn't supported against hadoop 17 but I am confident that it is likely broken in all versions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4680) fuse-dfs - df -kh on hdfs mount
shows much less %used than the dfs UI
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12648797#action_12648797 ]
Pete Wyckoff commented on HADOOP-4680:
--------------------------------------
just another note - this is only the % used seems to be off:
{code}
Filesystem Size Used Avail Use% Mounted on
fuse 20P 18P 1.1P 46% /export/hdfs
{code}
46% is clearly way off - should be more like 95% or so.
> fuse-dfs - df -kh on hdfs mount shows much less %used than the dfs UI
> ---------------------------------------------------------------------
>
> Key: HADOOP-4680
> URL: https://issues.apache.org/jira/browse/HADOOP-4680
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fuse-dfs
> Reporter: Pete Wyckoff
>
> i.e., statfs broken.
> This used to show the correct amount and now is showing wrong numbers in production environment.
> Yet, the unit test for this passes??
> This is the trunk version of fuse-dfs against 0.17 libhdfs and hadoop.
> Unknown if this affects 0.18.2 or 19.1 or trunk but i suspect it does.
> admittedly fuse-dfs isn't supported against hadoop 17 but I am confident that it is likely broken in all versions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.