You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "stack@archive.org (JIRA)" <ji...@apache.org> on 2006/04/28 01:05:37 UTC
[jira] Created: (NUTCH-256) Cannot open filename ....index.done.crc
Cannot open filename ....index.done.crc
---------------------------------------
Key: NUTCH-256
URL: http://issues.apache.org/jira/browse/NUTCH-256
Project: Nutch
Type: Bug
Components: indexer
Versions: 0.8-dev
Reporter: stack@archive.org
Priority: Minor
Trying to copy indices out of DFS I always get:
[bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
060427 160318 No FS indicated, using default:localhost:9001
060427 160318 Client connection to 127.0.0.1:9001: starting
060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done. Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:589)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-256) Cannot open filename
....index.done.crc
Posted by "stack@archive.org (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-256?page=comments#action_12376999 ]
stack@archive.org commented on NUTCH-256:
-----------------------------------------
Works for me. Thanks. Please close as fixed.
> Cannot open filename ....index.done.crc
> ---------------------------------------
>
> Key: NUTCH-256
> URL: http://issues.apache.org/jira/browse/NUTCH-256
> Project: Nutch
> Type: Bug
> Components: indexer
> Versions: 0.8-dev
> Reporter: stack@archive.org
> Priority: Minor
> Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done. Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
> at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:589)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-256) Cannot open filename
....index.done.crc
Posted by "stack@archive.org (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-256?page=comments#action_12376840 ]
stack@archive.org commented on NUTCH-256:
-----------------------------------------
Correct. Though it looks fatal, its not. Nor does it inspire confidence in the copy just made.
> Cannot open filename ....index.done.crc
> ---------------------------------------
>
> Key: NUTCH-256
> URL: http://issues.apache.org/jira/browse/NUTCH-256
> Project: Nutch
> Type: Bug
> Components: indexer
> Versions: 0.8-dev
> Reporter: stack@archive.org
> Priority: Minor
> Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done. Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
> at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:589)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (NUTCH-256) Cannot open filename
....index.done.crc
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-256?page=all ]
Doug Cutting resolved NUTCH-256:
--------------------------------
Resolution: Fixed
Assign To: Doug Cutting
This is fixed in Hadoop 0.2.
> Cannot open filename ....index.done.crc
> ---------------------------------------
>
> Key: NUTCH-256
> URL: http://issues.apache.org/jira/browse/NUTCH-256
> Project: Nutch
> Type: Bug
> Components: indexer
> Versions: 0.8-dev
> Reporter: stack@archive.org
> Assignee: Doug Cutting
> Priority: Minor
> Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done. Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
> at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:589)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-256) Cannot open filename
....index.done.crc
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-256?page=comments#action_12376839 ]
Doug Cutting commented on NUTCH-256:
------------------------------------
That's not a fatal exception, right? Everything still works? It should. This is just the DFS version of FileNotFound, which we otherwise ignore. Perhaps we should fix DFS to really throw a FileNotFound in this case, or we should just change this to a LOG.fine()?
> Cannot open filename ....index.done.crc
> ---------------------------------------
>
> Key: NUTCH-256
> URL: http://issues.apache.org/jira/browse/NUTCH-256
> Project: Nutch
> Type: Bug
> Components: indexer
> Versions: 0.8-dev
> Reporter: stack@archive.org
> Priority: Minor
> Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done. Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
> at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:589)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-256) Cannot open filename
....index.done.crc
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-256?page=comments#action_12376993 ]
Doug Cutting commented on NUTCH-256:
------------------------------------
I think this is really a bug in Hadoop's FileSystem.createNewFile() method. I've just fixed that. Does that work for you? If so, I'll close this.
> Cannot open filename ....index.done.crc
> ---------------------------------------
>
> Key: NUTCH-256
> URL: http://issues.apache.org/jira/browse/NUTCH-256
> Project: Nutch
> Type: Bug
> Components: indexer
> Versions: 0.8-dev
> Reporter: stack@archive.org
> Priority: Minor
> Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done. Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
> at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:589)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (NUTCH-256) Cannot open filename ....index.done.crc
Posted by "stack@archive.org (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-256?page=all ]
stack@archive.org updated NUTCH-256:
------------------------------------
Attachment: index.done.crc.patch
Ensure creation of companion index.done .crc file
> Cannot open filename ....index.done.crc
> ---------------------------------------
>
> Key: NUTCH-256
> URL: http://issues.apache.org/jira/browse/NUTCH-256
> Project: Nutch
> Type: Bug
> Components: indexer
> Versions: 0.8-dev
> Reporter: stack@archive.org
> Priority: Minor
> Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done. Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
> at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:589)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira