You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Stefan Groschupf (JIRA)" <ji...@apache.org> on 2005/11/18 16:37:41 UTC
[jira] Created: (NUTCH-127) uncorrect values using -du, or ls does not return items
uncorrect values using -du, or ls does not return items
-------------------------------------------------------
Key: NUTCH-127
URL: http://issues.apache.org/jira/browse/NUTCH-127
Project: Nutch
Type: Bug
Components: ndfs
Versions: 0.8-dev, 0.7.2-dev
Reporter: Stefan Groschupf
Priority: Blocker
The ndfs client return uncorrect values by using du or ls does not return items.
It looks like there is a problem with the virtual file strcuture, since -du only reads the meta data, isn't it?
We had moved some data from folder to folder and after that we notice that a folder with zero items has a size.
[nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/
051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
051118 092409 No FS indicated, using default:192.168.200.3:50000
051118 092409 Client connection to 192.168.200.3:50000: starting
Found 1 items
/user/nutch/indexes/20051022033721 974606348
[nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/20051022033721/
051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
051118 092416 No FS indicated, using default:192.168.200.3:50000
051118 092416 Client connection to 192.168.200.3:50000: starting
Found 0 items
[nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -ls indexes/20051022033721
051118 093331 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
051118 093332 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
051118 093332 No FS indicated, using default:192.168.200.3:50000
051118 093332 Client connection to 192.168.200.3:50000: starting
Found 0 items
So may the mv tool has a problem, the du or the ls tool. :-O Any ideas where to search for the problem? Dubugging ndfs is tricky.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (NUTCH-127) uncorrect values using -du, or ls does
not return items
Posted by "Stefan Groschupf (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-127?page=all ]
Stefan Groschupf resolved NUTCH-127:
------------------------------------
Resolution: Fixed
I guess it is solved, thanks. If able to reproduce it again I will just reopen this or a new report.
Thanks!
> uncorrect values using -du, or ls does not return items
> -------------------------------------------------------
>
> Key: NUTCH-127
> URL: http://issues.apache.org/jira/browse/NUTCH-127
> Project: Nutch
> Type: Bug
> Components: ndfs
> Versions: 0.8-dev, 0.7.2-dev
> Reporter: Stefan Groschupf
> Priority: Blocker
>
> The ndfs client return uncorrect values by using du or ls does not return items.
> It looks like there is a problem with the virtual file strcuture, since -du only reads the meta data, isn't it?
> We had moved some data from folder to folder and after that we notice that a folder with zero items has a size.
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/
> 051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 092409 No FS indicated, using default:192.168.200.3:50000
> 051118 092409 Client connection to 192.168.200.3:50000: starting
> Found 1 items
> /user/nutch/indexes/20051022033721 974606348
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/20051022033721/
> 051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 092416 No FS indicated, using default:192.168.200.3:50000
> 051118 092416 Client connection to 192.168.200.3:50000: starting
> Found 0 items
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -ls indexes/20051022033721
> 051118 093331 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 093332 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 093332 No FS indicated, using default:192.168.200.3:50000
> 051118 093332 Client connection to 192.168.200.3:50000: starting
> Found 0 items
> So may the mv tool has a problem, the du or the ls tool. :-O Any ideas where to search for the problem? Dubugging ndfs is tricky.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-127) uncorrect values using -du, or ls
does not return items
Posted by "Mike Cafarella (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-127?page=comments#action_12363613 ]
Mike Cafarella commented on NUTCH-127:
--------------------------------------
Stefan,
I fixed a namespace problem in NDFS just before Christmas. It sounds like your
issue falls into the set of things I fixed.
You mention the "mv" tool. The problem I fixed was: if you tried to move certain
directories, the move would fail and the dir you tried to move would disappear.
If that was your problem, then we can close this bug!
> uncorrect values using -du, or ls does not return items
> -------------------------------------------------------
>
> Key: NUTCH-127
> URL: http://issues.apache.org/jira/browse/NUTCH-127
> Project: Nutch
> Type: Bug
> Components: ndfs
> Versions: 0.8-dev, 0.7.2-dev
> Reporter: Stefan Groschupf
> Priority: Blocker
>
> The ndfs client return uncorrect values by using du or ls does not return items.
> It looks like there is a problem with the virtual file strcuture, since -du only reads the meta data, isn't it?
> We had moved some data from folder to folder and after that we notice that a folder with zero items has a size.
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/
> 051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 092409 No FS indicated, using default:192.168.200.3:50000
> 051118 092409 Client connection to 192.168.200.3:50000: starting
> Found 1 items
> /user/nutch/indexes/20051022033721 974606348
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/20051022033721/
> 051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 092416 No FS indicated, using default:192.168.200.3:50000
> 051118 092416 Client connection to 192.168.200.3:50000: starting
> Found 0 items
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -ls indexes/20051022033721
> 051118 093331 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 093332 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 093332 No FS indicated, using default:192.168.200.3:50000
> 051118 093332 Client connection to 192.168.200.3:50000: starting
> Found 0 items
> So may the mv tool has a problem, the du or the ls tool. :-O Any ideas where to search for the problem? Dubugging ndfs is tricky.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira