You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Dennis Kubes (JIRA)" <ji...@apache.org> on 2007/02/02 20:00:05 UTC
[jira] Created: (NUTCH-437) MapFile in Hadoop 0.10.2 has changed,
must update references
MapFile in Hadoop 0.10.2 has changed, must update references
------------------------------------------------------------
Key: NUTCH-437
URL: https://issues.apache.org/jira/browse/NUTCH-437
Project: Nutch
Issue Type: Bug
Affects Versions: 0.8.2, 0.9.0
Environment: windows xp and java
Reporter: Dennis Kubes
Assigned To: Dennis Kubes
Fix For: 0.8.2, 0.9.0
The MapFile.Writer signature has changed in hadoop 0.10.2 to include a Configuration object. Object in the Nutch codebase that reference MapFile.Writer will need to be updated.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (NUTCH-437) MapFile in Hadoop Trunk has changed,
must update references
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrzej Bialecki reassigned NUTCH-437:
---------------------------------------
Assignee: Andrzej Bialecki (was: Dennis Kubes)
> MapFile in Hadoop Trunk has changed, must update references
> -----------------------------------------------------------
>
> Key: NUTCH-437
> URL: https://issues.apache.org/jira/browse/NUTCH-437
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 0.8.2, 0.9.0
> Environment: windows xp and java
> Reporter: Dennis Kubes
> Assigned To: Andrzej Bialecki
> Fix For: 0.8.2, 0.9.0
>
> Attachments: nutch-hadoop-0.10.2-mapfile.patch
>
>
> The MapFile.Writer signature has changed in hadoop trunk (version 10.x +) to include a Configuration object. Object in the Nutch codebase that reference MapFile.Writer will need to be updated.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-437) MapFile in Hadoop Trunk has changed,
must update references
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-437:
-------------------------------
Description: The MapFile.Writer signature has changed in hadoop trunk (version 10.x +) to include a Configuration object. Object in the Nutch codebase that reference MapFile.Writer will need to be updated. (was: The MapFile.Writer signature has changed in hadoop 0.10.2 to include a Configuration object. Object in the Nutch codebase that reference MapFile.Writer will need to be updated. )
Summary: MapFile in Hadoop Trunk has changed, must update references (was: MapFile in Hadoop 0.10.2 has changed, must update references)
Updated issue to reflect changed Hadoop trunk version.
> MapFile in Hadoop Trunk has changed, must update references
> -----------------------------------------------------------
>
> Key: NUTCH-437
> URL: https://issues.apache.org/jira/browse/NUTCH-437
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 0.8.2, 0.9.0
> Environment: windows xp and java
> Reporter: Dennis Kubes
> Assigned To: Dennis Kubes
> Fix For: 0.8.2, 0.9.0
>
> Attachments: nutch-hadoop-0.10.2-mapfile.patch
>
>
> The MapFile.Writer signature has changed in hadoop trunk (version 10.x +) to include a Configuration object. Object in the Nutch codebase that reference MapFile.Writer will need to be updated.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-437) MapFile in Hadoop Trunk has changed,
must update references
Posted by "Armel Nene (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473064 ]
Armel Nene commented on NUTCH-437:
----------------------------------
I was wondering if this patch could fix my problem which is, if not the same, very similar to this one. I am using Nutch 0.8.2-dev, I have made checkout awhile ago from SVN but never updated again. I was able to crawl 10000 xml files before with no error whatsoever. This is the following errors that I get when I'm fetching:
INFO parser.custom: Custom-parse: Parsing content file:/C:/TeamBinder/AddressBook/9100/(65)E110_ST A0 (1).pdf
07/02/12 22:09:16 INFO fetcher.Fetcher: fetch of file:/C:/TeamBinder/AddressBook/9100/(65)E110_ST A0 (1).pdf failed with: java.lang.NullPointerException
07/02/12 22:09:17 INFO mapred.LocalJobRunner: 0 pages, 0 errors, 0.0 pages/s, 0 kb/s,
07/02/12 22:09:17 FATAL fetcher.Fetcher: java.lang.NullPointerException
07/02/12 22:09:17 FATAL fetcher.Fetcher: at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:198)
07/02/12 22:09:17 FATAL fetcher.Fetcher: at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:189)
07/02/12 22:09:17 FATAL fetcher.Fetcher: at org.apache.hadoop.mapred.MapTask$2.collect(MapTask.java:91)
07/02/12 22:09:17 FATAL fetcher.Fetcher: at org.apache.nutch.fetcher.Fetcher$FetcherThread.output(Fetcher.java:314)
07/02/12 22:09:17 FATAL fetcher.Fetcher: at org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:232)
07/02/12 22:09:17 FATAL fetcher.Fetcher: fetcher caught:java.lang.NullPointerException
One of the problem is that my hadoop version says the following: hadoop-0.4.0-patched. Now I don't know if it means that I am running the 0.4.0 version but it seems a little bit confusing. Once you can clarify that for me, then I will be able to apply the patch to my version.
Best Regards,
Armel
> MapFile in Hadoop Trunk has changed, must update references
> -----------------------------------------------------------
>
> Key: NUTCH-437
> URL: https://issues.apache.org/jira/browse/NUTCH-437
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 0.8.2, 0.9.0
> Environment: windows xp and java
> Reporter: Dennis Kubes
> Assigned To: Andrzej Bialecki
> Fix For: 0.8.2, 0.9.0
>
> Attachments: nutch-hadoop-0.10.2-mapfile.patch
>
>
> The MapFile.Writer signature has changed in hadoop trunk (version 10.x +) to include a Configuration object. Object in the Nutch codebase that reference MapFile.Writer will need to be updated.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-437) MapFile in Hadoop 0.10.2 has changed,
must update references
Posted by "stack@archive.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12472841 ]
stack@archive.org commented on NUTCH-437:
-----------------------------------------
+1. I reviewed and applied patch along with a hadoop-0.11.1-core jar in place of the current hadoop-0.10.1. Ran unit tests and all passed.
Might want to change this issues subject. Says 0.10.2 hadoop. Implication is patch only works with this (non-existent) hadoop.
> MapFile in Hadoop 0.10.2 has changed, must update references
> ------------------------------------------------------------
>
> Key: NUTCH-437
> URL: https://issues.apache.org/jira/browse/NUTCH-437
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 0.8.2, 0.9.0
> Environment: windows xp and java
> Reporter: Dennis Kubes
> Assigned To: Dennis Kubes
> Fix For: 0.8.2, 0.9.0
>
> Attachments: nutch-hadoop-0.10.2-mapfile.patch
>
>
> The MapFile.Writer signature has changed in hadoop 0.10.2 to include a Configuration object. Object in the Nutch codebase that reference MapFile.Writer will need to be updated.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (NUTCH-437) MapFile in Hadoop Trunk has changed,
must update references
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrzej Bialecki closed NUTCH-437.
-----------------------------------
Resolution: Fixed
Fix Version/s: (was: 0.8.2)
Fixed in rev. 515791 as part of the Hadoop upgrade.
> MapFile in Hadoop Trunk has changed, must update references
> -----------------------------------------------------------
>
> Key: NUTCH-437
> URL: https://issues.apache.org/jira/browse/NUTCH-437
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 0.8.2, 0.9.0
> Environment: windows xp and java
> Reporter: Dennis Kubes
> Assigned To: Andrzej Bialecki
> Fix For: 0.9.0
>
> Attachments: nutch-hadoop-0.10.2-mapfile.patch
>
>
> The MapFile.Writer signature has changed in hadoop trunk (version 10.x +) to include a Configuration object. Object in the Nutch codebase that reference MapFile.Writer will need to be updated.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-437) MapFile in Hadoop 0.10.2 has changed,
must update references
Posted by "Dennis Kubes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Kubes updated NUTCH-437:
-------------------------------
Attachment: nutch-hadoop-0.10.2-mapfile.patch
This patch changes the references to MapFile.Writer throughout the Nutch codebase to be compatible with the Hadoop 0.10.2 changes to MapFile.Writer. No new unit tests were needed these were handled through previous unit tests. Old unit tests were also updated to the new signature.
This patch should only be applied when upgrading to Hadoop 0.10.2
> MapFile in Hadoop 0.10.2 has changed, must update references
> ------------------------------------------------------------
>
> Key: NUTCH-437
> URL: https://issues.apache.org/jira/browse/NUTCH-437
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 0.8.2, 0.9.0
> Environment: windows xp and java
> Reporter: Dennis Kubes
> Assigned To: Dennis Kubes
> Fix For: 0.8.2, 0.9.0
>
> Attachments: nutch-hadoop-0.10.2-mapfile.patch
>
>
> The MapFile.Writer signature has changed in hadoop 0.10.2 to include a Configuration object. Object in the Nutch codebase that reference MapFile.Writer will need to be updated.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-437) MapFile in Hadoop Trunk has changed,
must update references
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12472848 ]
Andrzej Bialecki commented on NUTCH-437:
-----------------------------------------
I'm planning to update Hadoop to release 0.11.2 when it's released (next week). I'll also fix this issue then.
> MapFile in Hadoop Trunk has changed, must update references
> -----------------------------------------------------------
>
> Key: NUTCH-437
> URL: https://issues.apache.org/jira/browse/NUTCH-437
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 0.8.2, 0.9.0
> Environment: windows xp and java
> Reporter: Dennis Kubes
> Assigned To: Andrzej Bialecki
> Fix For: 0.8.2, 0.9.0
>
> Attachments: nutch-hadoop-0.10.2-mapfile.patch
>
>
> The MapFile.Writer signature has changed in hadoop trunk (version 10.x +) to include a Configuration object. Object in the Nutch codebase that reference MapFile.Writer will need to be updated.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.