You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Vipul Bharakhada <vi...@gmail.com> on 2012/03/16 19:38:32 UTC
_temporary doesn't exist
Hi,
I am using the old hadoop version 0.17.2 and I am getting the following
exception when I am trying to run a Job. But it only happens at particular
time, as cron jobs do run those task at particular intervals but it only
fails at one particular time in day.
Mar 14 06:49:23 7 08884: java.io.IOException: The directory
hdfs://<{IPADDRESS}>:<{PORT}>/myserver/matcher/output/_temporary doesnt
exist
Mar 14 06:49:23 7 08884: \09at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:1439)
Mar 14 06:49:23 7 08884: \09at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:1511)
Mar 14 06:49:23 7 08884: \09at
org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:723)
Mar 14 06:49:23 7 08884: \09at
org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
Mar 14 06:49:23 7 08884: \09at
org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
Mar 14 06:49:23 7 08884: \09at
org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
Mar 14 06:49:23 7 08884: \09at
org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
Mar 14 06:49:23 7 08884: \09at
org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
What can be a problem? as this folder is created by hadoop internally and
used internally, clean up is also done internally by hadoop. So why this
directory is missing at some particular time?
Any clue?
-Vipul
Re: _temporary doesn't exist
Posted by Deepak Nettem <de...@gmail.com>.
In the logs directory in your hadoop setup. You will have NameNode,
JobTracker and TaskTracker logs. Individual task tracker logs are generated
on the nodes.
You can also see the job specific logs from one of the web UIs if you're
using Apache Hadoop.
On Fri, Mar 16, 2012 at 5:04 PM, Vipul Bharakhada <vi...@gmail.com>wrote:
> Where can I find those logs?
> -Vipul
>
> On Fri, Mar 16, 2012 at 1:33 PM, Bejoy Ks <be...@gmail.com> wrote:
>
> > Hi Vipul
> > AFAIK, the clean up should happen after the job completion not task.
> > But what is causing this clean up, may be you can get some info from the
> NN
> > logs.
> >
> > Regards
> > Bejoy
> >
> > On Sat, Mar 17, 2012 at 12:48 AM, Vipul Bharakhada <vipulrb29@gmail.com
> > >wrote:
> >
> > > Its one of the server which I don't have permission to upgrade. but any
> > > help is fine. I saw a bug filed on 0.20 that mapreduce doesnt check for
> > the
> > > _temporary directory existence. but not so sure is it hadoop or its one
> > of
> > > the scheduled jobs on server which is cleaning up hadoop. I am curious
> > > about why its deleting the _temporary folder before the task is
> finished.
> > > the clean up should happen at the task completion If I am not wrong.
> > > Correct me If I am wrong. I am new to hadoop.
> > > Thank you.
> > > -Vipul
> > >
> > > On Fri, Mar 16, 2012 at 12:09 PM, Bejoy Ks <be...@gmail.com>
> > wrote:
> > >
> > > > Hi Vipul
> > > > Is there any reason you are with 0.17 version of hadoop? It is a
> > > > pretty old version of hadoop (more than 2 years back) and tons of bug
> > > fixes
> > > > and optimizations have gone into trunk after the same. You should
> badly
> > > > upgrade to any 1.0.X releases. It would be hard for any one on the
> list
> > > to
> > > > help you out with such an outdated version,Try an upgrade and
> > > > see whether this issue still persists.
> > > >
> > > > Regards
> > > > Bejoy
> > > >
> > > > On Sat, Mar 17, 2012 at 12:27 AM, Vipul Bharakhada <
> > vipulrb29@gmail.com
> > > > >wrote:
> > > >
> > > > > One more observation: usually this job takes 3 to 4 minutes,
> however
> > > when
> > > > > it fails, at that particular time it takes more than 42 to 50
> > minutes.
> > > > > -Vipul
> > > > >
> > > > > On Fri, Mar 16, 2012 at 11:38 AM, Vipul Bharakhada <
> > > vipulrb29@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Hi,
> > > > > > I am using the old hadoop version 0.17.2 and I am getting the
> > > following
> > > > > > exception when I am trying to run a Job. But it only happens at
> > > > > particular
> > > > > > time, as cron jobs do run those task at particular intervals but
> it
> > > > only
> > > > > > fails at one particular time in day.
> > > > > >
> > > > > > Mar 14 06:49:23 7 08884: java.io.IOException: The directory
> > > > > > hdfs://<{IPADDRESS}>:<{PORT}>/myserver/matcher/output/_temporary
> > > doesnt
> > > > > > exist
> > > > > > Mar 14 06:49:23 7 08884: \09at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:1439)
> > > > > > Mar 14 06:49:23 7 08884: \09at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:1511)
> > > > > > Mar 14 06:49:23 7 08884: \09at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:723)
> > > > > > Mar 14 06:49:23 7 08884: \09at
> > > > > >
> > > org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
> > > > > > Mar 14 06:49:23 7 08884: \09at
> > > > > >
> > > >
> > org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
> > > > > > Mar 14 06:49:23 7 08884: \09at
> > > > > >
> > > org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
> > > > > > Mar 14 06:49:23 7 08884: \09at
> > > > > > org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
> > > > > > Mar 14 06:49:23 7 08884: \09at
> > > > > > org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
> > > > > >
> > > > > > What can be a problem? as this folder is created by hadoop
> > internally
> > > > and
> > > > > > used internally, clean up is also done internally by hadoop. So
> why
> > > > this
> > > > > > directory is missing at some particular time?
> > > > > > Any clue?
> > > > > > -Vipul
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: _temporary doesn't exist
Posted by Vipul Bharakhada <vi...@gmail.com>.
Where can I find those logs?
-Vipul
On Fri, Mar 16, 2012 at 1:33 PM, Bejoy Ks <be...@gmail.com> wrote:
> Hi Vipul
> AFAIK, the clean up should happen after the job completion not task.
> But what is causing this clean up, may be you can get some info from the NN
> logs.
>
> Regards
> Bejoy
>
> On Sat, Mar 17, 2012 at 12:48 AM, Vipul Bharakhada <vipulrb29@gmail.com
> >wrote:
>
> > Its one of the server which I don't have permission to upgrade. but any
> > help is fine. I saw a bug filed on 0.20 that mapreduce doesnt check for
> the
> > _temporary directory existence. but not so sure is it hadoop or its one
> of
> > the scheduled jobs on server which is cleaning up hadoop. I am curious
> > about why its deleting the _temporary folder before the task is finished.
> > the clean up should happen at the task completion If I am not wrong.
> > Correct me If I am wrong. I am new to hadoop.
> > Thank you.
> > -Vipul
> >
> > On Fri, Mar 16, 2012 at 12:09 PM, Bejoy Ks <be...@gmail.com>
> wrote:
> >
> > > Hi Vipul
> > > Is there any reason you are with 0.17 version of hadoop? It is a
> > > pretty old version of hadoop (more than 2 years back) and tons of bug
> > fixes
> > > and optimizations have gone into trunk after the same. You should badly
> > > upgrade to any 1.0.X releases. It would be hard for any one on the list
> > to
> > > help you out with such an outdated version,Try an upgrade and
> > > see whether this issue still persists.
> > >
> > > Regards
> > > Bejoy
> > >
> > > On Sat, Mar 17, 2012 at 12:27 AM, Vipul Bharakhada <
> vipulrb29@gmail.com
> > > >wrote:
> > >
> > > > One more observation: usually this job takes 3 to 4 minutes, however
> > when
> > > > it fails, at that particular time it takes more than 42 to 50
> minutes.
> > > > -Vipul
> > > >
> > > > On Fri, Mar 16, 2012 at 11:38 AM, Vipul Bharakhada <
> > vipulrb29@gmail.com
> > > > >wrote:
> > > >
> > > > > Hi,
> > > > > I am using the old hadoop version 0.17.2 and I am getting the
> > following
> > > > > exception when I am trying to run a Job. But it only happens at
> > > > particular
> > > > > time, as cron jobs do run those task at particular intervals but it
> > > only
> > > > > fails at one particular time in day.
> > > > >
> > > > > Mar 14 06:49:23 7 08884: java.io.IOException: The directory
> > > > > hdfs://<{IPADDRESS}>:<{PORT}>/myserver/matcher/output/_temporary
> > doesnt
> > > > > exist
> > > > > Mar 14 06:49:23 7 08884: \09at
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:1439)
> > > > > Mar 14 06:49:23 7 08884: \09at
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:1511)
> > > > > Mar 14 06:49:23 7 08884: \09at
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:723)
> > > > > Mar 14 06:49:23 7 08884: \09at
> > > > >
> > org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
> > > > > Mar 14 06:49:23 7 08884: \09at
> > > > >
> > >
> org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
> > > > > Mar 14 06:49:23 7 08884: \09at
> > > > >
> > org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
> > > > > Mar 14 06:49:23 7 08884: \09at
> > > > > org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
> > > > > Mar 14 06:49:23 7 08884: \09at
> > > > > org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
> > > > >
> > > > > What can be a problem? as this folder is created by hadoop
> internally
> > > and
> > > > > used internally, clean up is also done internally by hadoop. So why
> > > this
> > > > > directory is missing at some particular time?
> > > > > Any clue?
> > > > > -Vipul
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: _temporary doesn't exist
Posted by Bejoy Ks <be...@gmail.com>.
Hi Vipul
AFAIK, the clean up should happen after the job completion not task.
But what is causing this clean up, may be you can get some info from the NN
logs.
Regards
Bejoy
On Sat, Mar 17, 2012 at 12:48 AM, Vipul Bharakhada <vi...@gmail.com>wrote:
> Its one of the server which I don't have permission to upgrade. but any
> help is fine. I saw a bug filed on 0.20 that mapreduce doesnt check for the
> _temporary directory existence. but not so sure is it hadoop or its one of
> the scheduled jobs on server which is cleaning up hadoop. I am curious
> about why its deleting the _temporary folder before the task is finished.
> the clean up should happen at the task completion If I am not wrong.
> Correct me If I am wrong. I am new to hadoop.
> Thank you.
> -Vipul
>
> On Fri, Mar 16, 2012 at 12:09 PM, Bejoy Ks <be...@gmail.com> wrote:
>
> > Hi Vipul
> > Is there any reason you are with 0.17 version of hadoop? It is a
> > pretty old version of hadoop (more than 2 years back) and tons of bug
> fixes
> > and optimizations have gone into trunk after the same. You should badly
> > upgrade to any 1.0.X releases. It would be hard for any one on the list
> to
> > help you out with such an outdated version,Try an upgrade and
> > see whether this issue still persists.
> >
> > Regards
> > Bejoy
> >
> > On Sat, Mar 17, 2012 at 12:27 AM, Vipul Bharakhada <vipulrb29@gmail.com
> > >wrote:
> >
> > > One more observation: usually this job takes 3 to 4 minutes, however
> when
> > > it fails, at that particular time it takes more than 42 to 50 minutes.
> > > -Vipul
> > >
> > > On Fri, Mar 16, 2012 at 11:38 AM, Vipul Bharakhada <
> vipulrb29@gmail.com
> > > >wrote:
> > >
> > > > Hi,
> > > > I am using the old hadoop version 0.17.2 and I am getting the
> following
> > > > exception when I am trying to run a Job. But it only happens at
> > > particular
> > > > time, as cron jobs do run those task at particular intervals but it
> > only
> > > > fails at one particular time in day.
> > > >
> > > > Mar 14 06:49:23 7 08884: java.io.IOException: The directory
> > > > hdfs://<{IPADDRESS}>:<{PORT}>/myserver/matcher/output/_temporary
> doesnt
> > > > exist
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:1439)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:1511)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:723)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> > org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > > org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > > org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
> > > >
> > > > What can be a problem? as this folder is created by hadoop internally
> > and
> > > > used internally, clean up is also done internally by hadoop. So why
> > this
> > > > directory is missing at some particular time?
> > > > Any clue?
> > > > -Vipul
> > > >
> > > >
> > >
> >
>
Re: _temporary doesn't exist
Posted by Vipul Bharakhada <vi...@gmail.com>.
Its one of the server which I don't have permission to upgrade. but any
help is fine. I saw a bug filed on 0.20 that mapreduce doesnt check for the
_temporary directory existence. but not so sure is it hadoop or its one of
the scheduled jobs on server which is cleaning up hadoop. I am curious
about why its deleting the _temporary folder before the task is finished.
the clean up should happen at the task completion If I am not wrong.
Correct me If I am wrong. I am new to hadoop.
Thank you.
-Vipul
On Fri, Mar 16, 2012 at 12:09 PM, Bejoy Ks <be...@gmail.com> wrote:
> Hi Vipul
> Is there any reason you are with 0.17 version of hadoop? It is a
> pretty old version of hadoop (more than 2 years back) and tons of bug fixes
> and optimizations have gone into trunk after the same. You should badly
> upgrade to any 1.0.X releases. It would be hard for any one on the list to
> help you out with such an outdated version,Try an upgrade and
> see whether this issue still persists.
>
> Regards
> Bejoy
>
> On Sat, Mar 17, 2012 at 12:27 AM, Vipul Bharakhada <vipulrb29@gmail.com
> >wrote:
>
> > One more observation: usually this job takes 3 to 4 minutes, however when
> > it fails, at that particular time it takes more than 42 to 50 minutes.
> > -Vipul
> >
> > On Fri, Mar 16, 2012 at 11:38 AM, Vipul Bharakhada <vipulrb29@gmail.com
> > >wrote:
> >
> > > Hi,
> > > I am using the old hadoop version 0.17.2 and I am getting the following
> > > exception when I am trying to run a Job. But it only happens at
> > particular
> > > time, as cron jobs do run those task at particular intervals but it
> only
> > > fails at one particular time in day.
> > >
> > > Mar 14 06:49:23 7 08884: java.io.IOException: The directory
> > > hdfs://<{IPADDRESS}>:<{PORT}>/myserver/matcher/output/_temporary doesnt
> > > exist
> > > Mar 14 06:49:23 7 08884: \09at
> > >
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:1439)
> > > Mar 14 06:49:23 7 08884: \09at
> > >
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:1511)
> > > Mar 14 06:49:23 7 08884: \09at
> > >
> >
> org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:723)
> > > Mar 14 06:49:23 7 08884: \09at
> > > org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
> > > Mar 14 06:49:23 7 08884: \09at
> > >
> org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
> > > Mar 14 06:49:23 7 08884: \09at
> > > org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
> > > Mar 14 06:49:23 7 08884: \09at
> > > org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
> > > Mar 14 06:49:23 7 08884: \09at
> > > org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
> > >
> > > What can be a problem? as this folder is created by hadoop internally
> and
> > > used internally, clean up is also done internally by hadoop. So why
> this
> > > directory is missing at some particular time?
> > > Any clue?
> > > -Vipul
> > >
> > >
> >
>
Re: _temporary doesn't exist
Posted by Bejoy Ks <be...@gmail.com>.
Hi Vipul
Is there any reason you are with 0.17 version of hadoop? It is a
pretty old version of hadoop (more than 2 years back) and tons of bug fixes
and optimizations have gone into trunk after the same. You should badly
upgrade to any 1.0.X releases. It would be hard for any one on the list to
help you out with such an outdated version,Try an upgrade and
see whether this issue still persists.
Regards
Bejoy
On Sat, Mar 17, 2012 at 12:27 AM, Vipul Bharakhada <vi...@gmail.com>wrote:
> One more observation: usually this job takes 3 to 4 minutes, however when
> it fails, at that particular time it takes more than 42 to 50 minutes.
> -Vipul
>
> On Fri, Mar 16, 2012 at 11:38 AM, Vipul Bharakhada <vipulrb29@gmail.com
> >wrote:
>
> > Hi,
> > I am using the old hadoop version 0.17.2 and I am getting the following
> > exception when I am trying to run a Job. But it only happens at
> particular
> > time, as cron jobs do run those task at particular intervals but it only
> > fails at one particular time in day.
> >
> > Mar 14 06:49:23 7 08884: java.io.IOException: The directory
> > hdfs://<{IPADDRESS}>:<{PORT}>/myserver/matcher/output/_temporary doesnt
> > exist
> > Mar 14 06:49:23 7 08884: \09at
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:1439)
> > Mar 14 06:49:23 7 08884: \09at
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:1511)
> > Mar 14 06:49:23 7 08884: \09at
> >
> org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:723)
> > Mar 14 06:49:23 7 08884: \09at
> > org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
> > Mar 14 06:49:23 7 08884: \09at
> > org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
> > Mar 14 06:49:23 7 08884: \09at
> > org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
> > Mar 14 06:49:23 7 08884: \09at
> > org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
> > Mar 14 06:49:23 7 08884: \09at
> > org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
> >
> > What can be a problem? as this folder is created by hadoop internally and
> > used internally, clean up is also done internally by hadoop. So why this
> > directory is missing at some particular time?
> > Any clue?
> > -Vipul
> >
> >
>
Re: _temporary doesn't exist
Posted by Vipul Bharakhada <vi...@gmail.com>.
One more observation: usually this job takes 3 to 4 minutes, however when
it fails, at that particular time it takes more than 42 to 50 minutes.
-Vipul
On Fri, Mar 16, 2012 at 11:38 AM, Vipul Bharakhada <vi...@gmail.com>wrote:
> Hi,
> I am using the old hadoop version 0.17.2 and I am getting the following
> exception when I am trying to run a Job. But it only happens at particular
> time, as cron jobs do run those task at particular intervals but it only
> fails at one particular time in day.
>
> Mar 14 06:49:23 7 08884: java.io.IOException: The directory
> hdfs://<{IPADDRESS}>:<{PORT}>/myserver/matcher/output/_temporary doesnt
> exist
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:1439)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:1511)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:723)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
>
> What can be a problem? as this folder is created by hadoop internally and
> used internally, clean up is also done internally by hadoop. So why this
> directory is missing at some particular time?
> Any clue?
> -Vipul
>
>
Re: _temporary doesn't exist
Posted by Vipul Bharakhada <vi...@gmail.com>.
One more observation: usually this job takes 3 to 4 minutes, however when
it fails, at that particular time it takes more than 42 to 50 minutes.
-Vipul
On Fri, Mar 16, 2012 at 11:38 AM, Vipul Bharakhada <vi...@gmail.com>wrote:
> Hi,
> I am using the old hadoop version 0.17.2 and I am getting the following
> exception when I am trying to run a Job. But it only happens at particular
> time, as cron jobs do run those task at particular intervals but it only
> fails at one particular time in day.
>
> Mar 14 06:49:23 7 08884: java.io.IOException: The directory
> hdfs://<{IPADDRESS}>:<{PORT}>/myserver/matcher/output/_temporary doesnt
> exist
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:1439)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:1511)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:723)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
> Mar 14 06:49:23 7 08884: \09at
> org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
>
> What can be a problem? as this folder is created by hadoop internally and
> used internally, clean up is also done internally by hadoop. So why this
> directory is missing at some particular time?
> Any clue?
> -Vipul
>
>