You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Lewis John Mcgibbney <le...@gmail.com> on 2013/02/28 21:06:44 UTC

Something for the weekend

Hi,
I pushed a real simple script which I use as a cron job to bootsrtrap
Apache Nutch with 1M URLs every day.
For those wanting to crawl, test, use Apache Nutch, I suppose this is a
decent way to get up and running.
https://github.com/lewismc/nipt
I thought I would share it with yous.
Thank you
Lewis

-- 
*Lewis*

Re: Something for the weekend

Posted by feng lu <am...@gmail.com>.

Hi Lewis

thanks for share Lewis.
But when i run the bootstrap.sh. it throw an warn.

rmdir: failed to remove `./temp': Directory not empty

Maybe we should comment this command
rmdir $temp

Thanks Lewis.



On Fri, Mar 1, 2013 at 6:04 AM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> Hi Tejas,
> I should probably say that I would like to thank Stanford University for
> allowing me to run my Nutch server(s) there ;) Just don't tell my professor
> I am working on this ;)
>
> On Thu, Feb 28, 2013 at 1:33 PM, Tejas Patil <tejas.patil.cs@gmail.com
> >wrote:
>
> > Thanks Lewis for sharing it. You must have an awesome cluster to run this
> > :)
> >
> > Thanks,
> > Tejas Patil
> >
> > On Thu, Feb 28, 2013 at 12:06 PM, Lewis John Mcgibbney <
> > lewis.mcgibbney@gmail.com> wrote:
> >
> > > Hi,
> > > I pushed a real simple script which I use as a cron job to bootsrtrap
> > > Apache Nutch with 1M URLs every day.
> > > For those wanting to crawl, test, use Apache Nutch, I suppose this is a
> > > decent way to get up and running.
> > > https://github.com/lewismc/nipt
> > > I thought I would share it with yous.
> > > Thank you
> > > Lewis
> > >
> > > --
> > > *Lewis*
> > >
> >
>
>
>
> --
> *Lewis*
>



-- 
Don't Grow Old, Grow Up... :-)

Re: Something for the weekend

Posted by Lewis John Mcgibbney <le...@gmail.com>.

Hi Tejas,
I should probably say that I would like to thank Stanford University for
allowing me to run my Nutch server(s) there ;) Just don't tell my professor
I am working on this ;)

On Thu, Feb 28, 2013 at 1:33 PM, Tejas Patil <te...@gmail.com>wrote:

> Thanks Lewis for sharing it. You must have an awesome cluster to run this
> :)
>
> Thanks,
> Tejas Patil
>
> On Thu, Feb 28, 2013 at 12:06 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > Hi,
> > I pushed a real simple script which I use as a cron job to bootsrtrap
> > Apache Nutch with 1M URLs every day.
> > For those wanting to crawl, test, use Apache Nutch, I suppose this is a
> > decent way to get up and running.
> > https://github.com/lewismc/nipt
> > I thought I would share it with yous.
> > Thank you
> > Lewis
> >
> > --
> > *Lewis*
> >
>



-- 
*Lewis*

Re: Something for the weekend

Posted by Tejas Patil <te...@gmail.com>.

Thanks Lewis for sharing it. You must have an awesome cluster to run this :)

Thanks,
Tejas Patil

On Thu, Feb 28, 2013 at 12:06 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> Hi,
> I pushed a real simple script which I use as a cron job to bootsrtrap
> Apache Nutch with 1M URLs every day.
> For those wanting to crawl, test, use Apache Nutch, I suppose this is a
> decent way to get up and running.
> https://github.com/lewismc/nipt
> I thought I would share it with yous.
> Thank you
> Lewis
>
> --
> *Lewis*
>

Re: Something for the weekend

Posted by Renato Marroquín Mogrovejo <re...@gmail.com>.

Cool! I will definitely use this to play with Apache Nutch.
Thanks Lewis!


Renato M.

2013/2/28 Lewis John Mcgibbney <le...@gmail.com>:
> Hi,
> I pushed a real simple script which I use as a cron job to bootsrtrap
> Apache Nutch with 1M URLs every day.
> For those wanting to crawl, test, use Apache Nutch, I suppose this is a
> decent way to get up and running.
> https://github.com/lewismc/nipt
> I thought I would share it with yous.
> Thank you
> Lewis
>
> --
> *Lewis*