You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Srinivas Gokavarapu <sr...@gmail.com> on 2008/09/16 07:20:43 UTC

Re: Temporary storage during crawling

Hi,
          I am crawling large data from the web. I have started crawling and
I got an error saying no disk space. After checking I came to know that
nutch stores temporarily during crawling in /tmp folder. I dont have much
space in / directory. But I have more space on my /home2 directory where I
started the crawling. Can somebody help to change the temporary storage to
some folder in my home2 for crawling.

Thanx in advance,
Srinivas.

Re: Temporary storage during crawling

Posted by Srinivas Gokavarapu <sr...@gmail.com>.

hi,
       Thank you very much. It worked fine.

On Tue, Sep 16, 2008 at 10:58 AM, Susam Pal <su...@gmail.com> wrote:

> You can use the 'hadoop.tmp.dir' property in hadoop-site.xml to
> specify an alternate path for temporary directory.
>
> Example:
>
> <property>
>  <name>hadoop.tmp.dir</name>
>  <value>/home2/tmp/</value>
>  <description></description>
> </property>
>
> Regards,
> Susam Pal
>
> On Tue, Sep 16, 2008 at 10:50 AM, Srinivas Gokavarapu
> <sr...@gmail.com> wrote:
> > Hi,
> >          I am crawling large data from the web. I have started crawling
> and
> > I got an error saying no disk space. After checking I came to know that
> > nutch stores temporarily during crawling in /tmp folder. I dont have much
> > space in / directory. But I have more space on my /home2 directory where
> I
> > started the crawling. Can somebody help to change the temporary storage
> to
> > some folder in my home2 for crawling.
> >
> > Thanx in advance,
> > Srinivas.
> >
>

Re: Temporary storage during crawling

Posted by Susam Pal <su...@gmail.com>.

You can use the 'hadoop.tmp.dir' property in hadoop-site.xml to
specify an alternate path for temporary directory.

Example:

<property>
  <name>hadoop.tmp.dir</name>
  <value>/home2/tmp/</value>
  <description></description>
</property>

Regards,
Susam Pal

On Tue, Sep 16, 2008 at 10:50 AM, Srinivas Gokavarapu
<sr...@gmail.com> wrote:
> Hi,
>          I am crawling large data from the web. I have started crawling and
> I got an error saying no disk space. After checking I came to know that
> nutch stores temporarily during crawling in /tmp folder. I dont have much
> space in / directory. But I have more space on my /home2 directory where I
> started the crawling. Can somebody help to change the temporary storage to
> some folder in my home2 for crawling.
>
> Thanx in advance,
> Srinivas.
>