You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Boyu Zhang <bo...@gmail.com> on 2013/06/07 22:33:08 UTC
Job History files location of 2.0.4
Dear All,
I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
old job history files (used to be in hdfs, output/_logs/history), it
records detailed time information for every task attempts.
But now it is not on hdfs anymore, I copied the entire "/" from hdfs to my
local dir, but am not able to find this location.
Could anyone give any advise on where are the files?
Thanks,
Boyu
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
Thanks for letting us know the solution.
Regards,
Shahab
On Mon, Jun 10, 2013 at 3:40 PM, Boyu Zhang <bo...@gmail.com> wrote:
> I solved the problem, in my case, it is because I did not start the job
> history server daemon. After starting it, the history logs are generated in
> 2 places: the HADOOP_MAPRED_LOG_DIR directory, and in hdfs, /tmp folder.
>
> Hope this is helpful for whoever also has the same problem. By the way,
> the documentation for hadoop 2.0.x is out of date, this creates some
> trouble in figuring out where the conf/logs are stored and where to set
> them.
>
> Thanks,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 2:39 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> I used a directory that is local to every slave node: export
>> HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
>>
>> I did not change the " hadoop.job.history.user.location", I thought if I
>> don't change this property, the job history is going to be stored in hdfs
>> under output/_logs dir.
>>
>> Then after the job completes, I copied back the logs to the server.
>>
>> Thanks a lot,
>> Boyu
>>
>>
>> On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>
>>> What value do you have for hadoop.log.dir property?
>>>
>>>
>>> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Hi Shahab,
>>>>
>>>>
>>>> How old were they?
>>>>>
>>>> They are new, I did the copy automatically right after the job
>>>> completed, in a script.
>>>>
>>>> I am assuming they were from the jobs run on the older version, right?
>>>>>
>>>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>>>
>>>> Or are you looking for new jobs's log that you are running after the
>>>>> upgrade?
>>>>>
>>>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>>>> then run the job, then copied back the entire hdfs directory.
>>>>
>>>>
>>>>> What about the local file systems? Are the logs there still?
>>>>>
>>>> Where should I find the logs (job logs, not daemon logs) in the local
>>>> file systems?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>>
>>>>>
>>>>> Regards,
>>>>> Shahab
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>
>>>>>> Thanks Shahab,
>>>>>>
>>>>>> I saw the link, but it is not the case for me. I copied everything
>>>>>> from hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did
>>>>>> not see the logs.
>>>>>>
>>>>>> Did it work for you?
>>>>>>
>>>>>> Thanks,
>>>>>> Boyu
>>>>>>
>>>>>>
>>>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>>>
>>>>>>> See this;
>>>>>>>
>>>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>>>
>>>>>>> Regards,
>>>>>>> Shahab
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>>>
>>>>>>>> Dear All,
>>>>>>>>
>>>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to
>>>>>>>> find the old job history files (used to be in hdfs, output/_logs/history),
>>>>>>>> it records detailed time information for every task attempts.
>>>>>>>>
>>>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from
>>>>>>>> hdfs to my local dir, but am not able to find this location.
>>>>>>>>
>>>>>>>> Could anyone give any advise on where are the files?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Boyu
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
Thanks for letting us know the solution.
Regards,
Shahab
On Mon, Jun 10, 2013 at 3:40 PM, Boyu Zhang <bo...@gmail.com> wrote:
> I solved the problem, in my case, it is because I did not start the job
> history server daemon. After starting it, the history logs are generated in
> 2 places: the HADOOP_MAPRED_LOG_DIR directory, and in hdfs, /tmp folder.
>
> Hope this is helpful for whoever also has the same problem. By the way,
> the documentation for hadoop 2.0.x is out of date, this creates some
> trouble in figuring out where the conf/logs are stored and where to set
> them.
>
> Thanks,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 2:39 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> I used a directory that is local to every slave node: export
>> HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
>>
>> I did not change the " hadoop.job.history.user.location", I thought if I
>> don't change this property, the job history is going to be stored in hdfs
>> under output/_logs dir.
>>
>> Then after the job completes, I copied back the logs to the server.
>>
>> Thanks a lot,
>> Boyu
>>
>>
>> On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>
>>> What value do you have for hadoop.log.dir property?
>>>
>>>
>>> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Hi Shahab,
>>>>
>>>>
>>>> How old were they?
>>>>>
>>>> They are new, I did the copy automatically right after the job
>>>> completed, in a script.
>>>>
>>>> I am assuming they were from the jobs run on the older version, right?
>>>>>
>>>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>>>
>>>> Or are you looking for new jobs's log that you are running after the
>>>>> upgrade?
>>>>>
>>>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>>>> then run the job, then copied back the entire hdfs directory.
>>>>
>>>>
>>>>> What about the local file systems? Are the logs there still?
>>>>>
>>>> Where should I find the logs (job logs, not daemon logs) in the local
>>>> file systems?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>>
>>>>>
>>>>> Regards,
>>>>> Shahab
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>
>>>>>> Thanks Shahab,
>>>>>>
>>>>>> I saw the link, but it is not the case for me. I copied everything
>>>>>> from hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did
>>>>>> not see the logs.
>>>>>>
>>>>>> Did it work for you?
>>>>>>
>>>>>> Thanks,
>>>>>> Boyu
>>>>>>
>>>>>>
>>>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>>>
>>>>>>> See this;
>>>>>>>
>>>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>>>
>>>>>>> Regards,
>>>>>>> Shahab
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>>>
>>>>>>>> Dear All,
>>>>>>>>
>>>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to
>>>>>>>> find the old job history files (used to be in hdfs, output/_logs/history),
>>>>>>>> it records detailed time information for every task attempts.
>>>>>>>>
>>>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from
>>>>>>>> hdfs to my local dir, but am not able to find this location.
>>>>>>>>
>>>>>>>> Could anyone give any advise on where are the files?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Boyu
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
Thanks for letting us know the solution.
Regards,
Shahab
On Mon, Jun 10, 2013 at 3:40 PM, Boyu Zhang <bo...@gmail.com> wrote:
> I solved the problem, in my case, it is because I did not start the job
> history server daemon. After starting it, the history logs are generated in
> 2 places: the HADOOP_MAPRED_LOG_DIR directory, and in hdfs, /tmp folder.
>
> Hope this is helpful for whoever also has the same problem. By the way,
> the documentation for hadoop 2.0.x is out of date, this creates some
> trouble in figuring out where the conf/logs are stored and where to set
> them.
>
> Thanks,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 2:39 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> I used a directory that is local to every slave node: export
>> HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
>>
>> I did not change the " hadoop.job.history.user.location", I thought if I
>> don't change this property, the job history is going to be stored in hdfs
>> under output/_logs dir.
>>
>> Then after the job completes, I copied back the logs to the server.
>>
>> Thanks a lot,
>> Boyu
>>
>>
>> On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>
>>> What value do you have for hadoop.log.dir property?
>>>
>>>
>>> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Hi Shahab,
>>>>
>>>>
>>>> How old were they?
>>>>>
>>>> They are new, I did the copy automatically right after the job
>>>> completed, in a script.
>>>>
>>>> I am assuming they were from the jobs run on the older version, right?
>>>>>
>>>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>>>
>>>> Or are you looking for new jobs's log that you are running after the
>>>>> upgrade?
>>>>>
>>>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>>>> then run the job, then copied back the entire hdfs directory.
>>>>
>>>>
>>>>> What about the local file systems? Are the logs there still?
>>>>>
>>>> Where should I find the logs (job logs, not daemon logs) in the local
>>>> file systems?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>>
>>>>>
>>>>> Regards,
>>>>> Shahab
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>
>>>>>> Thanks Shahab,
>>>>>>
>>>>>> I saw the link, but it is not the case for me. I copied everything
>>>>>> from hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did
>>>>>> not see the logs.
>>>>>>
>>>>>> Did it work for you?
>>>>>>
>>>>>> Thanks,
>>>>>> Boyu
>>>>>>
>>>>>>
>>>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>>>
>>>>>>> See this;
>>>>>>>
>>>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>>>
>>>>>>> Regards,
>>>>>>> Shahab
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>>>
>>>>>>>> Dear All,
>>>>>>>>
>>>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to
>>>>>>>> find the old job history files (used to be in hdfs, output/_logs/history),
>>>>>>>> it records detailed time information for every task attempts.
>>>>>>>>
>>>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from
>>>>>>>> hdfs to my local dir, but am not able to find this location.
>>>>>>>>
>>>>>>>> Could anyone give any advise on where are the files?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Boyu
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
Thanks for letting us know the solution.
Regards,
Shahab
On Mon, Jun 10, 2013 at 3:40 PM, Boyu Zhang <bo...@gmail.com> wrote:
> I solved the problem, in my case, it is because I did not start the job
> history server daemon. After starting it, the history logs are generated in
> 2 places: the HADOOP_MAPRED_LOG_DIR directory, and in hdfs, /tmp folder.
>
> Hope this is helpful for whoever also has the same problem. By the way,
> the documentation for hadoop 2.0.x is out of date, this creates some
> trouble in figuring out where the conf/logs are stored and where to set
> them.
>
> Thanks,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 2:39 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> I used a directory that is local to every slave node: export
>> HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
>>
>> I did not change the " hadoop.job.history.user.location", I thought if I
>> don't change this property, the job history is going to be stored in hdfs
>> under output/_logs dir.
>>
>> Then after the job completes, I copied back the logs to the server.
>>
>> Thanks a lot,
>> Boyu
>>
>>
>> On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>
>>> What value do you have for hadoop.log.dir property?
>>>
>>>
>>> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Hi Shahab,
>>>>
>>>>
>>>> How old were they?
>>>>>
>>>> They are new, I did the copy automatically right after the job
>>>> completed, in a script.
>>>>
>>>> I am assuming they were from the jobs run on the older version, right?
>>>>>
>>>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>>>
>>>> Or are you looking for new jobs's log that you are running after the
>>>>> upgrade?
>>>>>
>>>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>>>> then run the job, then copied back the entire hdfs directory.
>>>>
>>>>
>>>>> What about the local file systems? Are the logs there still?
>>>>>
>>>> Where should I find the logs (job logs, not daemon logs) in the local
>>>> file systems?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>>
>>>>>
>>>>> Regards,
>>>>> Shahab
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>
>>>>>> Thanks Shahab,
>>>>>>
>>>>>> I saw the link, but it is not the case for me. I copied everything
>>>>>> from hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did
>>>>>> not see the logs.
>>>>>>
>>>>>> Did it work for you?
>>>>>>
>>>>>> Thanks,
>>>>>> Boyu
>>>>>>
>>>>>>
>>>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>>>
>>>>>>> See this;
>>>>>>>
>>>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>>>
>>>>>>> Regards,
>>>>>>> Shahab
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>>>
>>>>>>>> Dear All,
>>>>>>>>
>>>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to
>>>>>>>> find the old job history files (used to be in hdfs, output/_logs/history),
>>>>>>>> it records detailed time information for every task attempts.
>>>>>>>>
>>>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from
>>>>>>>> hdfs to my local dir, but am not able to find this location.
>>>>>>>>
>>>>>>>> Could anyone give any advise on where are the files?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Boyu
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
I solved the problem, in my case, it is because I did not start the job
history server daemon. After starting it, the history logs are generated in
2 places: the HADOOP_MAPRED_LOG_DIR directory, and in hdfs, /tmp folder.
Hope this is helpful for whoever also has the same problem. By the way, the
documentation for hadoop 2.0.x is out of date, this creates some trouble in
figuring out where the conf/logs are stored and where to set them.
Thanks,
Boyu
On Fri, Jun 7, 2013 at 2:39 PM, Boyu Zhang <bo...@gmail.com> wrote:
> I used a directory that is local to every slave node: export
> HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
>
> I did not change the " hadoop.job.history.user.location", I thought if I
> don't change this property, the job history is going to be stored in hdfs
> under output/_logs dir.
>
> Then after the job completes, I copied back the logs to the server.
>
> Thanks a lot,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> What value do you have for hadoop.log.dir property?
>>
>>
>> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Hi Shahab,
>>>
>>>
>>> How old were they?
>>>>
>>> They are new, I did the copy automatically right after the job
>>> completed, in a script.
>>>
>>> I am assuming they were from the jobs run on the older version, right?
>>>>
>>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>>
>>> Or are you looking for new jobs's log that you are running after the
>>>> upgrade?
>>>>
>>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>>> then run the job, then copied back the entire hdfs directory.
>>>
>>>
>>>> What about the local file systems? Are the logs there still?
>>>>
>>> Where should I find the logs (job logs, not daemon logs) in the local
>>> file systems?
>>>
>>> Thanks,
>>> Boyu
>>>
>>>
>>>>
>>>> Regards,
>>>> Shahab
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>
>>>>> Thanks Shahab,
>>>>>
>>>>> I saw the link, but it is not the case for me. I copied everything
>>>>> from hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did
>>>>> not see the logs.
>>>>>
>>>>> Did it work for you?
>>>>>
>>>>> Thanks,
>>>>> Boyu
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>>
>>>>>> See this;
>>>>>>
>>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>>
>>>>>> Regards,
>>>>>> Shahab
>>>>>>
>>>>>>
>>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>>
>>>>>>> Dear All,
>>>>>>>
>>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>>>> records detailed time information for every task attempts.
>>>>>>>
>>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>>>> to my local dir, but am not able to find this location.
>>>>>>>
>>>>>>> Could anyone give any advise on where are the files?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Boyu
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
I solved the problem, in my case, it is because I did not start the job
history server daemon. After starting it, the history logs are generated in
2 places: the HADOOP_MAPRED_LOG_DIR directory, and in hdfs, /tmp folder.
Hope this is helpful for whoever also has the same problem. By the way, the
documentation for hadoop 2.0.x is out of date, this creates some trouble in
figuring out where the conf/logs are stored and where to set them.
Thanks,
Boyu
On Fri, Jun 7, 2013 at 2:39 PM, Boyu Zhang <bo...@gmail.com> wrote:
> I used a directory that is local to every slave node: export
> HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
>
> I did not change the " hadoop.job.history.user.location", I thought if I
> don't change this property, the job history is going to be stored in hdfs
> under output/_logs dir.
>
> Then after the job completes, I copied back the logs to the server.
>
> Thanks a lot,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> What value do you have for hadoop.log.dir property?
>>
>>
>> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Hi Shahab,
>>>
>>>
>>> How old were they?
>>>>
>>> They are new, I did the copy automatically right after the job
>>> completed, in a script.
>>>
>>> I am assuming they were from the jobs run on the older version, right?
>>>>
>>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>>
>>> Or are you looking for new jobs's log that you are running after the
>>>> upgrade?
>>>>
>>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>>> then run the job, then copied back the entire hdfs directory.
>>>
>>>
>>>> What about the local file systems? Are the logs there still?
>>>>
>>> Where should I find the logs (job logs, not daemon logs) in the local
>>> file systems?
>>>
>>> Thanks,
>>> Boyu
>>>
>>>
>>>>
>>>> Regards,
>>>> Shahab
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>
>>>>> Thanks Shahab,
>>>>>
>>>>> I saw the link, but it is not the case for me. I copied everything
>>>>> from hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did
>>>>> not see the logs.
>>>>>
>>>>> Did it work for you?
>>>>>
>>>>> Thanks,
>>>>> Boyu
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>>
>>>>>> See this;
>>>>>>
>>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>>
>>>>>> Regards,
>>>>>> Shahab
>>>>>>
>>>>>>
>>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>>
>>>>>>> Dear All,
>>>>>>>
>>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>>>> records detailed time information for every task attempts.
>>>>>>>
>>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>>>> to my local dir, but am not able to find this location.
>>>>>>>
>>>>>>> Could anyone give any advise on where are the files?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Boyu
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
I solved the problem, in my case, it is because I did not start the job
history server daemon. After starting it, the history logs are generated in
2 places: the HADOOP_MAPRED_LOG_DIR directory, and in hdfs, /tmp folder.
Hope this is helpful for whoever also has the same problem. By the way, the
documentation for hadoop 2.0.x is out of date, this creates some trouble in
figuring out where the conf/logs are stored and where to set them.
Thanks,
Boyu
On Fri, Jun 7, 2013 at 2:39 PM, Boyu Zhang <bo...@gmail.com> wrote:
> I used a directory that is local to every slave node: export
> HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
>
> I did not change the " hadoop.job.history.user.location", I thought if I
> don't change this property, the job history is going to be stored in hdfs
> under output/_logs dir.
>
> Then after the job completes, I copied back the logs to the server.
>
> Thanks a lot,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> What value do you have for hadoop.log.dir property?
>>
>>
>> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Hi Shahab,
>>>
>>>
>>> How old were they?
>>>>
>>> They are new, I did the copy automatically right after the job
>>> completed, in a script.
>>>
>>> I am assuming they were from the jobs run on the older version, right?
>>>>
>>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>>
>>> Or are you looking for new jobs's log that you are running after the
>>>> upgrade?
>>>>
>>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>>> then run the job, then copied back the entire hdfs directory.
>>>
>>>
>>>> What about the local file systems? Are the logs there still?
>>>>
>>> Where should I find the logs (job logs, not daemon logs) in the local
>>> file systems?
>>>
>>> Thanks,
>>> Boyu
>>>
>>>
>>>>
>>>> Regards,
>>>> Shahab
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>
>>>>> Thanks Shahab,
>>>>>
>>>>> I saw the link, but it is not the case for me. I copied everything
>>>>> from hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did
>>>>> not see the logs.
>>>>>
>>>>> Did it work for you?
>>>>>
>>>>> Thanks,
>>>>> Boyu
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>>
>>>>>> See this;
>>>>>>
>>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>>
>>>>>> Regards,
>>>>>> Shahab
>>>>>>
>>>>>>
>>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>>
>>>>>>> Dear All,
>>>>>>>
>>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>>>> records detailed time information for every task attempts.
>>>>>>>
>>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>>>> to my local dir, but am not able to find this location.
>>>>>>>
>>>>>>> Could anyone give any advise on where are the files?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Boyu
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
I solved the problem, in my case, it is because I did not start the job
history server daemon. After starting it, the history logs are generated in
2 places: the HADOOP_MAPRED_LOG_DIR directory, and in hdfs, /tmp folder.
Hope this is helpful for whoever also has the same problem. By the way, the
documentation for hadoop 2.0.x is out of date, this creates some trouble in
figuring out where the conf/logs are stored and where to set them.
Thanks,
Boyu
On Fri, Jun 7, 2013 at 2:39 PM, Boyu Zhang <bo...@gmail.com> wrote:
> I used a directory that is local to every slave node: export
> HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
>
> I did not change the " hadoop.job.history.user.location", I thought if I
> don't change this property, the job history is going to be stored in hdfs
> under output/_logs dir.
>
> Then after the job completes, I copied back the logs to the server.
>
> Thanks a lot,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> What value do you have for hadoop.log.dir property?
>>
>>
>> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Hi Shahab,
>>>
>>>
>>> How old were they?
>>>>
>>> They are new, I did the copy automatically right after the job
>>> completed, in a script.
>>>
>>> I am assuming they were from the jobs run on the older version, right?
>>>>
>>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>>
>>> Or are you looking for new jobs's log that you are running after the
>>>> upgrade?
>>>>
>>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>>> then run the job, then copied back the entire hdfs directory.
>>>
>>>
>>>> What about the local file systems? Are the logs there still?
>>>>
>>> Where should I find the logs (job logs, not daemon logs) in the local
>>> file systems?
>>>
>>> Thanks,
>>> Boyu
>>>
>>>
>>>>
>>>> Regards,
>>>> Shahab
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>
>>>>> Thanks Shahab,
>>>>>
>>>>> I saw the link, but it is not the case for me. I copied everything
>>>>> from hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did
>>>>> not see the logs.
>>>>>
>>>>> Did it work for you?
>>>>>
>>>>> Thanks,
>>>>> Boyu
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>>
>>>>>> See this;
>>>>>>
>>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>>
>>>>>> Regards,
>>>>>> Shahab
>>>>>>
>>>>>>
>>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>>
>>>>>>> Dear All,
>>>>>>>
>>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>>>> records detailed time information for every task attempts.
>>>>>>>
>>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>>>> to my local dir, but am not able to find this location.
>>>>>>>
>>>>>>> Could anyone give any advise on where are the files?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Boyu
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
I used a directory that is local to every slave node: export
HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
I did not change the " hadoop.job.history.user.location", I thought if I
don't change this property, the job history is going to be stored in hdfs
under output/_logs dir.
Then after the job completes, I copied back the logs to the server.
Thanks a lot,
Boyu
On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com> wrote:
> What value do you have for hadoop.log.dir property?
>
>
> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Hi Shahab,
>>
>>
>> How old were they?
>>>
>> They are new, I did the copy automatically right after the job completed,
>> in a script.
>>
>> I am assuming they were from the jobs run on the older version, right?
>>>
>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>
>> Or are you looking for new jobs's log that you are running after the
>>> upgrade?
>>>
>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>> then run the job, then copied back the entire hdfs directory.
>>
>>
>>> What about the local file systems? Are the logs there still?
>>>
>> Where should I find the logs (job logs, not daemon logs) in the local
>> file systems?
>>
>> Thanks,
>> Boyu
>>
>>
>>>
>>> Regards,
>>> Shahab
>>>
>>>
>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Thanks Shahab,
>>>>
>>>> I saw the link, but it is not the case for me. I copied everything from
>>>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>>>> the logs.
>>>>
>>>> Did it work for you?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>
>>>>> See this;
>>>>>
>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>
>>>>> Regards,
>>>>> Shahab
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>
>>>>>> Dear All,
>>>>>>
>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>>> records detailed time information for every task attempts.
>>>>>>
>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>>> to my local dir, but am not able to find this location.
>>>>>>
>>>>>> Could anyone give any advise on where are the files?
>>>>>>
>>>>>> Thanks,
>>>>>> Boyu
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
I used a directory that is local to every slave node: export
HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
I did not change the " hadoop.job.history.user.location", I thought if I
don't change this property, the job history is going to be stored in hdfs
under output/_logs dir.
Then after the job completes, I copied back the logs to the server.
Thanks a lot,
Boyu
On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com> wrote:
> What value do you have for hadoop.log.dir property?
>
>
> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Hi Shahab,
>>
>>
>> How old were they?
>>>
>> They are new, I did the copy automatically right after the job completed,
>> in a script.
>>
>> I am assuming they were from the jobs run on the older version, right?
>>>
>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>
>> Or are you looking for new jobs's log that you are running after the
>>> upgrade?
>>>
>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>> then run the job, then copied back the entire hdfs directory.
>>
>>
>>> What about the local file systems? Are the logs there still?
>>>
>> Where should I find the logs (job logs, not daemon logs) in the local
>> file systems?
>>
>> Thanks,
>> Boyu
>>
>>
>>>
>>> Regards,
>>> Shahab
>>>
>>>
>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Thanks Shahab,
>>>>
>>>> I saw the link, but it is not the case for me. I copied everything from
>>>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>>>> the logs.
>>>>
>>>> Did it work for you?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>
>>>>> See this;
>>>>>
>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>
>>>>> Regards,
>>>>> Shahab
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>
>>>>>> Dear All,
>>>>>>
>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>>> records detailed time information for every task attempts.
>>>>>>
>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>>> to my local dir, but am not able to find this location.
>>>>>>
>>>>>> Could anyone give any advise on where are the files?
>>>>>>
>>>>>> Thanks,
>>>>>> Boyu
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
I used a directory that is local to every slave node: export
HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
I did not change the " hadoop.job.history.user.location", I thought if I
don't change this property, the job history is going to be stored in hdfs
under output/_logs dir.
Then after the job completes, I copied back the logs to the server.
Thanks a lot,
Boyu
On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com> wrote:
> What value do you have for hadoop.log.dir property?
>
>
> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Hi Shahab,
>>
>>
>> How old were they?
>>>
>> They are new, I did the copy automatically right after the job completed,
>> in a script.
>>
>> I am assuming they were from the jobs run on the older version, right?
>>>
>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>
>> Or are you looking for new jobs's log that you are running after the
>>> upgrade?
>>>
>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>> then run the job, then copied back the entire hdfs directory.
>>
>>
>>> What about the local file systems? Are the logs there still?
>>>
>> Where should I find the logs (job logs, not daemon logs) in the local
>> file systems?
>>
>> Thanks,
>> Boyu
>>
>>
>>>
>>> Regards,
>>> Shahab
>>>
>>>
>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Thanks Shahab,
>>>>
>>>> I saw the link, but it is not the case for me. I copied everything from
>>>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>>>> the logs.
>>>>
>>>> Did it work for you?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>
>>>>> See this;
>>>>>
>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>
>>>>> Regards,
>>>>> Shahab
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>
>>>>>> Dear All,
>>>>>>
>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>>> records detailed time information for every task attempts.
>>>>>>
>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>>> to my local dir, but am not able to find this location.
>>>>>>
>>>>>> Could anyone give any advise on where are the files?
>>>>>>
>>>>>> Thanks,
>>>>>> Boyu
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
I used a directory that is local to every slave node: export
HADOOP_LOG_DIR="/scratch/$USER/$PBS_JOBID/hadoop-$USER/log".
I did not change the " hadoop.job.history.user.location", I thought if I
don't change this property, the job history is going to be stored in hdfs
under output/_logs dir.
Then after the job completes, I copied back the logs to the server.
Thanks a lot,
Boyu
On Fri, Jun 7, 2013 at 2:32 PM, Shahab Yunus <sh...@gmail.com> wrote:
> What value do you have for hadoop.log.dir property?
>
>
> On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Hi Shahab,
>>
>>
>> How old were they?
>>>
>> They are new, I did the copy automatically right after the job completed,
>> in a script.
>>
>> I am assuming they were from the jobs run on the older version, right?
>>>
>> I run the job using the hadoop version 2.0.4 if this is what you mean.
>>
>> Or are you looking for new jobs's log that you are running after the
>>> upgrade?
>>>
>> I am looking for new job's log, I did a fresh install of hadoop2.0.4,
>> then run the job, then copied back the entire hdfs directory.
>>
>>
>>> What about the local file systems? Are the logs there still?
>>>
>> Where should I find the logs (job logs, not daemon logs) in the local
>> file systems?
>>
>> Thanks,
>> Boyu
>>
>>
>>>
>>> Regards,
>>> Shahab
>>>
>>>
>>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Thanks Shahab,
>>>>
>>>> I saw the link, but it is not the case for me. I copied everything from
>>>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>>>> the logs.
>>>>
>>>> Did it work for you?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>>
>>>>> See this;
>>>>>
>>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>>
>>>>> Regards,
>>>>> Shahab
>>>>>
>>>>>
>>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>>
>>>>>> Dear All,
>>>>>>
>>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>>> records detailed time information for every task attempts.
>>>>>>
>>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>>> to my local dir, but am not able to find this location.
>>>>>>
>>>>>> Could anyone give any advise on where are the files?
>>>>>>
>>>>>> Thanks,
>>>>>> Boyu
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
What value do you have for hadoop.log.dir property?
On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Hi Shahab,
>
>
> How old were they?
>>
> They are new, I did the copy automatically right after the job completed,
> in a script.
>
> I am assuming they were from the jobs run on the older version, right?
>>
> I run the job using the hadoop version 2.0.4 if this is what you mean.
>
> Or are you looking for new jobs's log that you are running after the
>> upgrade?
>>
> I am looking for new job's log, I did a fresh install of hadoop2.0.4, then
> run the job, then copied back the entire hdfs directory.
>
>
>> What about the local file systems? Are the logs there still?
>>
> Where should I find the logs (job logs, not daemon logs) in the local file
> systems?
>
> Thanks,
> Boyu
>
>
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Thanks Shahab,
>>>
>>> I saw the link, but it is not the case for me. I copied everything from
>>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>>> the logs.
>>>
>>> Did it work for you?
>>>
>>> Thanks,
>>> Boyu
>>>
>>>
>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>
>>>> See this;
>>>>
>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>
>>>> Regards,
>>>> Shahab
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>
>>>>> Dear All,
>>>>>
>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>> records detailed time information for every task attempts.
>>>>>
>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>> to my local dir, but am not able to find this location.
>>>>>
>>>>> Could anyone give any advise on where are the files?
>>>>>
>>>>> Thanks,
>>>>> Boyu
>>>>>
>>>>
>>>>
>>>
>>
>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
What value do you have for hadoop.log.dir property?
On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Hi Shahab,
>
>
> How old were they?
>>
> They are new, I did the copy automatically right after the job completed,
> in a script.
>
> I am assuming they were from the jobs run on the older version, right?
>>
> I run the job using the hadoop version 2.0.4 if this is what you mean.
>
> Or are you looking for new jobs's log that you are running after the
>> upgrade?
>>
> I am looking for new job's log, I did a fresh install of hadoop2.0.4, then
> run the job, then copied back the entire hdfs directory.
>
>
>> What about the local file systems? Are the logs there still?
>>
> Where should I find the logs (job logs, not daemon logs) in the local file
> systems?
>
> Thanks,
> Boyu
>
>
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Thanks Shahab,
>>>
>>> I saw the link, but it is not the case for me. I copied everything from
>>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>>> the logs.
>>>
>>> Did it work for you?
>>>
>>> Thanks,
>>> Boyu
>>>
>>>
>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>
>>>> See this;
>>>>
>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>
>>>> Regards,
>>>> Shahab
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>
>>>>> Dear All,
>>>>>
>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>> records detailed time information for every task attempts.
>>>>>
>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>> to my local dir, but am not able to find this location.
>>>>>
>>>>> Could anyone give any advise on where are the files?
>>>>>
>>>>> Thanks,
>>>>> Boyu
>>>>>
>>>>
>>>>
>>>
>>
>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
What value do you have for hadoop.log.dir property?
On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Hi Shahab,
>
>
> How old were they?
>>
> They are new, I did the copy automatically right after the job completed,
> in a script.
>
> I am assuming they were from the jobs run on the older version, right?
>>
> I run the job using the hadoop version 2.0.4 if this is what you mean.
>
> Or are you looking for new jobs's log that you are running after the
>> upgrade?
>>
> I am looking for new job's log, I did a fresh install of hadoop2.0.4, then
> run the job, then copied back the entire hdfs directory.
>
>
>> What about the local file systems? Are the logs there still?
>>
> Where should I find the logs (job logs, not daemon logs) in the local file
> systems?
>
> Thanks,
> Boyu
>
>
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Thanks Shahab,
>>>
>>> I saw the link, but it is not the case for me. I copied everything from
>>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>>> the logs.
>>>
>>> Did it work for you?
>>>
>>> Thanks,
>>> Boyu
>>>
>>>
>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>
>>>> See this;
>>>>
>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>
>>>> Regards,
>>>> Shahab
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>
>>>>> Dear All,
>>>>>
>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>> records detailed time information for every task attempts.
>>>>>
>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>> to my local dir, but am not able to find this location.
>>>>>
>>>>> Could anyone give any advise on where are the files?
>>>>>
>>>>> Thanks,
>>>>> Boyu
>>>>>
>>>>
>>>>
>>>
>>
>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
What value do you have for hadoop.log.dir property?
On Fri, Jun 7, 2013 at 5:20 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Hi Shahab,
>
>
> How old were they?
>>
> They are new, I did the copy automatically right after the job completed,
> in a script.
>
> I am assuming they were from the jobs run on the older version, right?
>>
> I run the job using the hadoop version 2.0.4 if this is what you mean.
>
> Or are you looking for new jobs's log that you are running after the
>> upgrade?
>>
> I am looking for new job's log, I did a fresh install of hadoop2.0.4, then
> run the job, then copied back the entire hdfs directory.
>
>
>> What about the local file systems? Are the logs there still?
>>
> Where should I find the logs (job logs, not daemon logs) in the local file
> systems?
>
> Thanks,
> Boyu
>
>
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Thanks Shahab,
>>>
>>> I saw the link, but it is not the case for me. I copied everything from
>>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>>> the logs.
>>>
>>> Did it work for you?
>>>
>>> Thanks,
>>> Boyu
>>>
>>>
>>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>>
>>>> See this;
>>>>
>>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>>
>>>> Regards,
>>>> Shahab
>>>>
>>>>
>>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>>
>>>>> Dear All,
>>>>>
>>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>>> records detailed time information for every task attempts.
>>>>>
>>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs
>>>>> to my local dir, but am not able to find this location.
>>>>>
>>>>> Could anyone give any advise on where are the files?
>>>>>
>>>>> Thanks,
>>>>> Boyu
>>>>>
>>>>
>>>>
>>>
>>
>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
Hi Shahab,
How old were they?
>
They are new, I did the copy automatically right after the job completed,
in a script.
I am assuming they were from the jobs run on the older version, right?
>
I run the job using the hadoop version 2.0.4 if this is what you mean.
Or are you looking for new jobs's log that you are running after the
> upgrade?
>
I am looking for new job's log, I did a fresh install of hadoop2.0.4, then
run the job, then copied back the entire hdfs directory.
> What about the local file systems? Are the logs there still?
>
Where should I find the logs (job logs, not daemon logs) in the local file
systems?
Thanks,
Boyu
>
> Regards,
> Shahab
>
>
> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Thanks Shahab,
>>
>> I saw the link, but it is not the case for me. I copied everything from
>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>> the logs.
>>
>> Did it work for you?
>>
>> Thanks,
>> Boyu
>>
>>
>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>
>>> See this;
>>>
>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>
>>> Regards,
>>> Shahab
>>>
>>>
>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Dear All,
>>>>
>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>> records detailed time information for every task attempts.
>>>>
>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>>>> my local dir, but am not able to find this location.
>>>>
>>>> Could anyone give any advise on where are the files?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
Hi Shahab,
How old were they?
>
They are new, I did the copy automatically right after the job completed,
in a script.
I am assuming they were from the jobs run on the older version, right?
>
I run the job using the hadoop version 2.0.4 if this is what you mean.
Or are you looking for new jobs's log that you are running after the
> upgrade?
>
I am looking for new job's log, I did a fresh install of hadoop2.0.4, then
run the job, then copied back the entire hdfs directory.
> What about the local file systems? Are the logs there still?
>
Where should I find the logs (job logs, not daemon logs) in the local file
systems?
Thanks,
Boyu
>
> Regards,
> Shahab
>
>
> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Thanks Shahab,
>>
>> I saw the link, but it is not the case for me. I copied everything from
>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>> the logs.
>>
>> Did it work for you?
>>
>> Thanks,
>> Boyu
>>
>>
>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>
>>> See this;
>>>
>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>
>>> Regards,
>>> Shahab
>>>
>>>
>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Dear All,
>>>>
>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>> records detailed time information for every task attempts.
>>>>
>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>>>> my local dir, but am not able to find this location.
>>>>
>>>> Could anyone give any advise on where are the files?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
Hi Shahab,
How old were they?
>
They are new, I did the copy automatically right after the job completed,
in a script.
I am assuming they were from the jobs run on the older version, right?
>
I run the job using the hadoop version 2.0.4 if this is what you mean.
Or are you looking for new jobs's log that you are running after the
> upgrade?
>
I am looking for new job's log, I did a fresh install of hadoop2.0.4, then
run the job, then copied back the entire hdfs directory.
> What about the local file systems? Are the logs there still?
>
Where should I find the logs (job logs, not daemon logs) in the local file
systems?
Thanks,
Boyu
>
> Regards,
> Shahab
>
>
> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Thanks Shahab,
>>
>> I saw the link, but it is not the case for me. I copied everything from
>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>> the logs.
>>
>> Did it work for you?
>>
>> Thanks,
>> Boyu
>>
>>
>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>
>>> See this;
>>>
>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>
>>> Regards,
>>> Shahab
>>>
>>>
>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Dear All,
>>>>
>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>> records detailed time information for every task attempts.
>>>>
>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>>>> my local dir, but am not able to find this location.
>>>>
>>>> Could anyone give any advise on where are the files?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
Hi Shahab,
How old were they?
>
They are new, I did the copy automatically right after the job completed,
in a script.
I am assuming they were from the jobs run on the older version, right?
>
I run the job using the hadoop version 2.0.4 if this is what you mean.
Or are you looking for new jobs's log that you are running after the
> upgrade?
>
I am looking for new job's log, I did a fresh install of hadoop2.0.4, then
run the job, then copied back the entire hdfs directory.
> What about the local file systems? Are the logs there still?
>
Where should I find the logs (job logs, not daemon logs) in the local file
systems?
Thanks,
Boyu
>
> Regards,
> Shahab
>
>
> On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Thanks Shahab,
>>
>> I saw the link, but it is not the case for me. I copied everything from
>> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
>> the logs.
>>
>> Did it work for you?
>>
>> Thanks,
>> Boyu
>>
>>
>> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>>
>>> See this;
>>>
>>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>>
>>> Regards,
>>> Shahab
>>>
>>>
>>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com>wrote:
>>>
>>>> Dear All,
>>>>
>>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find
>>>> the old job history files (used to be in hdfs, output/_logs/history), it
>>>> records detailed time information for every task attempts.
>>>>
>>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>>>> my local dir, but am not able to find this location.
>>>>
>>>> Could anyone give any advise on where are the files?
>>>>
>>>> Thanks,
>>>> Boyu
>>>>
>>>
>>>
>>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
How old were they? What are the chances that they were past their retention
period and got deleted? I am assuming they were from the jobs run on the
older version, right? Or are you looking for new jobs's log that you are
running after the upgrade? What about the local file systems? Are the logs
there still?
Regards,
Shahab
On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Thanks Shahab,
>
> I saw the link, but it is not the case for me. I copied everything from
> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
> the logs.
>
> Did it work for you?
>
> Thanks,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> See this;
>>
>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Dear All,
>>>
>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
>>> old job history files (used to be in hdfs, output/_logs/history), it
>>> records detailed time information for every task attempts.
>>>
>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>>> my local dir, but am not able to find this location.
>>>
>>> Could anyone give any advise on where are the files?
>>>
>>> Thanks,
>>> Boyu
>>>
>>
>>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
How old were they? What are the chances that they were past their retention
period and got deleted? I am assuming they were from the jobs run on the
older version, right? Or are you looking for new jobs's log that you are
running after the upgrade? What about the local file systems? Are the logs
there still?
Regards,
Shahab
On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Thanks Shahab,
>
> I saw the link, but it is not the case for me. I copied everything from
> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
> the logs.
>
> Did it work for you?
>
> Thanks,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> See this;
>>
>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Dear All,
>>>
>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
>>> old job history files (used to be in hdfs, output/_logs/history), it
>>> records detailed time information for every task attempts.
>>>
>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>>> my local dir, but am not able to find this location.
>>>
>>> Could anyone give any advise on where are the files?
>>>
>>> Thanks,
>>> Boyu
>>>
>>
>>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
How old were they? What are the chances that they were past their retention
period and got deleted? I am assuming they were from the jobs run on the
older version, right? Or are you looking for new jobs's log that you are
running after the upgrade? What about the local file systems? Are the logs
there still?
Regards,
Shahab
On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Thanks Shahab,
>
> I saw the link, but it is not the case for me. I copied everything from
> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
> the logs.
>
> Did it work for you?
>
> Thanks,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> See this;
>>
>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Dear All,
>>>
>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
>>> old job history files (used to be in hdfs, output/_logs/history), it
>>> records detailed time information for every task attempts.
>>>
>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>>> my local dir, but am not able to find this location.
>>>
>>> Could anyone give any advise on where are the files?
>>>
>>> Thanks,
>>> Boyu
>>>
>>
>>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
How old were they? What are the chances that they were past their retention
period and got deleted? I am assuming they were from the jobs run on the
older version, right? Or are you looking for new jobs's log that you are
running after the upgrade? What about the local file systems? Are the logs
there still?
Regards,
Shahab
On Fri, Jun 7, 2013 at 4:56 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Thanks Shahab,
>
> I saw the link, but it is not the case for me. I copied everything from
> hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
> the logs.
>
> Did it work for you?
>
> Thanks,
> Boyu
>
>
> On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com>wrote:
>
>> See this;
>>
>> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>>
>> Regards,
>> Shahab
>>
>>
>> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
>>
>>> Dear All,
>>>
>>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
>>> old job history files (used to be in hdfs, output/_logs/history), it
>>> records detailed time information for every task attempts.
>>>
>>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>>> my local dir, but am not able to find this location.
>>>
>>> Could anyone give any advise on where are the files?
>>>
>>> Thanks,
>>> Boyu
>>>
>>
>>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
Thanks Shahab,
I saw the link, but it is not the case for me. I copied everything from
hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
the logs.
Did it work for you?
Thanks,
Boyu
On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com> wrote:
> See this;
>
> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>
> Regards,
> Shahab
>
>
> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Dear All,
>>
>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
>> old job history files (used to be in hdfs, output/_logs/history), it
>> records detailed time information for every task attempts.
>>
>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>> my local dir, but am not able to find this location.
>>
>> Could anyone give any advise on where are the files?
>>
>> Thanks,
>> Boyu
>>
>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
Thanks Shahab,
I saw the link, but it is not the case for me. I copied everything from
hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
the logs.
Did it work for you?
Thanks,
Boyu
On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com> wrote:
> See this;
>
> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>
> Regards,
> Shahab
>
>
> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Dear All,
>>
>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
>> old job history files (used to be in hdfs, output/_logs/history), it
>> records detailed time information for every task attempts.
>>
>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>> my local dir, but am not able to find this location.
>>
>> Could anyone give any advise on where are the files?
>>
>> Thanks,
>> Boyu
>>
>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
Thanks Shahab,
I saw the link, but it is not the case for me. I copied everything from
hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
the logs.
Did it work for you?
Thanks,
Boyu
On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com> wrote:
> See this;
>
> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>
> Regards,
> Shahab
>
>
> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Dear All,
>>
>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
>> old job history files (used to be in hdfs, output/_logs/history), it
>> records detailed time information for every task attempts.
>>
>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>> my local dir, but am not able to find this location.
>>
>> Could anyone give any advise on where are the files?
>>
>> Thanks,
>> Boyu
>>
>
>
Re: Job History files location of 2.0.4
Posted by Boyu Zhang <bo...@gmail.com>.
Thanks Shahab,
I saw the link, but it is not the case for me. I copied everything from
hdfs ($HADOOP_HOME/bin/hdfs dfs -copyToLocal / $local_dir). But did not see
the logs.
Did it work for you?
Thanks,
Boyu
On Fri, Jun 7, 2013 at 1:52 PM, Shahab Yunus <sh...@gmail.com> wrote:
> See this;
>
> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
>
> Regards,
> Shahab
>
>
> On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
>
>> Dear All,
>>
>> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
>> old job history files (used to be in hdfs, output/_logs/history), it
>> records detailed time information for every task attempts.
>>
>> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to
>> my local dir, but am not able to find this location.
>>
>> Could anyone give any advise on where are the files?
>>
>> Thanks,
>> Boyu
>>
>
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
See this;
http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
Regards,
Shahab
On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Dear All,
>
> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
> old job history files (used to be in hdfs, output/_logs/history), it
> records detailed time information for every task attempts.
>
> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to my
> local dir, but am not able to find this location.
>
> Could anyone give any advise on where are the files?
>
> Thanks,
> Boyu
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
See this;
http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
Regards,
Shahab
On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Dear All,
>
> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
> old job history files (used to be in hdfs, output/_logs/history), it
> records detailed time information for every task attempts.
>
> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to my
> local dir, but am not able to find this location.
>
> Could anyone give any advise on where are the files?
>
> Thanks,
> Boyu
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
See this;
http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
Regards,
Shahab
On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Dear All,
>
> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
> old job history files (used to be in hdfs, output/_logs/history), it
> records detailed time information for every task attempts.
>
> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to my
> local dir, but am not able to find this location.
>
> Could anyone give any advise on where are the files?
>
> Thanks,
> Boyu
>
Re: Job History files location of 2.0.4
Posted by Shahab Yunus <sh...@gmail.com>.
See this;
http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201302.mbox/%3C1360184802.61630.YahooMailNeo@web141205.mail.bf1.yahoo.com%3E
Regards,
Shahab
On Fri, Jun 7, 2013 at 4:33 PM, Boyu Zhang <bo...@gmail.com> wrote:
> Dear All,
>
> I recently moved from Hadoop0.20.2 to 2.0.4, and I am trying to find the
> old job history files (used to be in hdfs, output/_logs/history), it
> records detailed time information for every task attempts.
>
> But now it is not on hdfs anymore, I copied the entire "/" from hdfs to my
> local dir, but am not able to find this location.
>
> Could anyone give any advise on where are the files?
>
> Thanks,
> Boyu
>