You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Nutch User - 1 <nu...@gmail.com> on 2011/06/21 16:55:20 UTC
TestFetcher hangs
I followed the tutorial Testing Nutch 2.0 under Eclipse
(http://techvineyard.blogspot.com/2010/12/build-nutch-20.html) and ran
into a problem. The JUnit test TestFetcher doesn't complete. It hangs on
"map 100% reduce 0%". The entire output is here
(http://pastebin.com/m2XLjCTy).
At first it didn't even find suffix-urlfilter.txt as can be seen on the
line 93 of the previous output (http://pastebin.com/5tcieRXc). The
problem is probably related to Hadoop. Lines 7 and 8 may give a hint.
Do you have any ideas how to solve this?
Re: TestFetcher hangs
Posted by Nutch User - 1 <nu...@gmail.com>.
Oh yes, I forgot to tell that everything before TestFetcher JUnit test
went just fine. I used HSQL because of its easy installing procedures (I
don't have administrator's privileges on my workstation), and
TestGoraStorage JUnit test passed.
If my gora.properties is of interest, it can be found here:
(http://pastebin.com/shqsgXvH). As far as I know, it should be OK with
the HSQL setup I made.
On 07/01/2011 10:25 PM, Alexis wrote:
> Hi NutchUser1,
>
> It's hard to tell about your problem from the logs, so I'll just make
> assumptions here.
> If the test hangs at the initialization of the Fetch Hadoop job, it is
> probably due to an issue while connecting to your datastore.
>
> Which one are you using? MySQL, HBase?
>
> Can you please validate first you Gora setup by editing
> gora.properties file (which needs to be in the classpath) and then
> running the TestGoraStorage JUnit test. I recommend you apply the
> following patch first:
> http://techvineyard.blogspot.com/2010/12/build-nutch-20.html#Datastore
> before running the test
>
>
>
> Alexis
>
>
> On Tue, Jun 21, 2011 at 7:55 AM, Nutch User - 1 <nu...@gmail.com> wrote:
>> I followed the tutorial Testing Nutch 2.0 under Eclipse
>> (http://techvineyard.blogspot.com/2010/12/build-nutch-20.html) and ran
>> into a problem. The JUnit test TestFetcher doesn't complete. It hangs on
>> "map 100% reduce 0%". The entire output is here
>> (http://pastebin.com/m2XLjCTy).
>>
>> At first it didn't even find suffix-urlfilter.txt as can be seen on the
>> line 93 of the previous output (http://pastebin.com/5tcieRXc). The
>> problem is probably related to Hadoop. Lines 7 and 8 may give a hint.
>>
>> Do you have any ideas how to solve this?
>>
Re: TestFetcher hangs
Posted by Alexis <al...@gmail.com>.
Hi NutchUser1,
It's hard to tell about your problem from the logs, so I'll just make
assumptions here.
If the test hangs at the initialization of the Fetch Hadoop job, it is
probably due to an issue while connecting to your datastore.
Which one are you using? MySQL, HBase?
Can you please validate first you Gora setup by editing
gora.properties file (which needs to be in the classpath) and then
running the TestGoraStorage JUnit test. I recommend you apply the
following patch first:
http://techvineyard.blogspot.com/2010/12/build-nutch-20.html#Datastore
before running the test
Alexis
On Tue, Jun 21, 2011 at 7:55 AM, Nutch User - 1 <nu...@gmail.com> wrote:
> I followed the tutorial Testing Nutch 2.0 under Eclipse
> (http://techvineyard.blogspot.com/2010/12/build-nutch-20.html) and ran
> into a problem. The JUnit test TestFetcher doesn't complete. It hangs on
> "map 100% reduce 0%". The entire output is here
> (http://pastebin.com/m2XLjCTy).
>
> At first it didn't even find suffix-urlfilter.txt as can be seen on the
> line 93 of the previous output (http://pastebin.com/5tcieRXc). The
> problem is probably related to Hadoop. Lines 7 and 8 may give a hint.
>
> Do you have any ideas how to solve this?
>