You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by "Scheffel, Aaron" <Aa...@washpost.com> on 2013/09/05 14:18:44 UTC

Possible Bug Nutch 1.7 Crawl Script

The 1.7 crawl script has the following line 125

SEGMENT=`ls -l $CRAWL_PATH/segments/ | sed -e "s/ /\\n/g" | egrep 20[0-9]+ | sort -n | tail -n 1`

There is an "ls -l " there which is causing the script to behave badly (not work) at least on OS X. A simple "ls" seems to fix it.

-Aaron

Re: Possible Bug Nutch 1.7 Crawl Script

Posted by Sebastian Nagel <wa...@googlemail.com>.

Thanks Aaron, thanks Tejas,

opened NUTCH-1639 to keep this on radar.

Sebastian

On 09/09/2013 03:18 PM, Scheffel, Aaron wrote:
> Yep that works. 
> 
> Sorry, it took me a bit to get back around to testing it.
> -Aaron
> 
> On 9/5/13 10:26 AM, "Tejas Patil" <te...@gmail.com> wrote:
> 
>> Thanks for reporting that. I think this point was brought up over [0] but
>> was left off. Could you try out this and tell if it works ?
>> SEGMENT=`ls $CRAWL_PATH/segments/ | sort -n | tail -n 1`
>>
>> [0] :
>> https://issues.apache.org/jira/browse/NUTCH-1087?focusedCommentId=13554353
>> &page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#com
>> ment-13554353
>>
>>
>>
>> On Thu, Sep 5, 2013 at 5:18 AM, Scheffel, Aaron
>> <Aaron.Scheffel@washpost.com
>>> wrote:
>>
>>> The 1.7 crawl script has the following line 125
>>>
>>> SEGMENT=`ls -l $CRAWL_PATH/segments/ | sed -e "s/ /\\n/g" | egrep
>>> 20[0-9]+
>>> | sort -n | tail -n 1`
>>>
>>> There is an "ls -l " there which is causing the script to behave badly
>>> (not work) at least on OS X. A simple "ls" seems to fix it.
>>>
>>> -Aaron
>>>
>

Re: Possible Bug Nutch 1.7 Crawl Script

Posted by "Scheffel, Aaron" <Aa...@washpost.com>.

Yep that works. 

Sorry, it took me a bit to get back around to testing it.
-Aaron

On 9/5/13 10:26 AM, "Tejas Patil" <te...@gmail.com> wrote:

>Thanks for reporting that. I think this point was brought up over [0] but
>was left off. Could you try out this and tell if it works ?
>SEGMENT=`ls $CRAWL_PATH/segments/ | sort -n | tail -n 1`
>
>[0] :
>https://issues.apache.org/jira/browse/NUTCH-1087?focusedCommentId=13554353
>&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#com
>ment-13554353
>
>
>
>On Thu, Sep 5, 2013 at 5:18 AM, Scheffel, Aaron
><Aaron.Scheffel@washpost.com
>> wrote:
>
>> The 1.7 crawl script has the following line 125
>>
>> SEGMENT=`ls -l $CRAWL_PATH/segments/ | sed -e "s/ /\\n/g" | egrep
>>20[0-9]+
>> | sort -n | tail -n 1`
>>
>> There is an "ls -l " there which is causing the script to behave badly
>> (not work) at least on OS X. A simple "ls" seems to fix it.
>>
>> -Aaron
>>

Re: Possible Bug Nutch 1.7 Crawl Script

Posted by Tejas Patil <te...@gmail.com>.

Thanks for reporting that. I think this point was brought up over [0] but
was left off. Could you try out this and tell if it works ?
SEGMENT=`ls $CRAWL_PATH/segments/ | sort -n | tail -n 1`

[0] :
https://issues.apache.org/jira/browse/NUTCH-1087?focusedCommentId=13554353&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13554353



On Thu, Sep 5, 2013 at 5:18 AM, Scheffel, Aaron <Aaron.Scheffel@washpost.com
> wrote:

> The 1.7 crawl script has the following line 125
>
> SEGMENT=`ls -l $CRAWL_PATH/segments/ | sed -e "s/ /\\n/g" | egrep 20[0-9]+
> | sort -n | tail -n 1`
>
> There is an "ls -l " there which is causing the script to behave badly
> (not work) at least on OS X. A simple "ls" seems to fix it.
>
> -Aaron
>