You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Sayan Kole <sa...@gmail.com> on 2013/03/06 22:49:29 UTC
location of log files for mapreduce job run through eclipse
Hi,
I cannot find the log files for the wordcount job: job_local_0001 when
I run it through eclipse. I am getting the standard output on screen
showing me the progress of the maps/reduces, but when I check my log
directory inside the hadoop folder, I do not see any log files for this job
nor does the jobtracker log file have any information for this job.
Sayan Kole
Re: location of log files for mapreduce job run through eclipse
Posted by Sayan Kole <sa...@gmail.com>.
Ok. thanks. When I run the job from eclipse/command line, only certain
Log.info gets displayed on the console from certain files but not from
JobTracker. How do I display Log.info messages in JobTracker.java onto the
console like it does say for example Task.java shown below
13/03/07 12:31:17 INFO mapred.Task: Task 'attempt_local_0001_m_000000_0'
done.
13/03/07 12:31:17 INFO mapred.Task: Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@bdccedd
Thanks,
Sayan
On Thu, Mar 7, 2013 at 12:07 PM, Abdelrhman Shettia <
ashettia@hortonworks.com> wrote:
> Hi Sayan ,
>
> May want to try to use the log4j in hadoop and place it with the
> packages/jars and validate it matches the one in the cluster or print it to
> the console . I came across nice document that can help you in this case.
>
>
> http://www.thecloudavenue.com/2012/10/debugging-hadoop-mapreduce-program-in.html
>
>
> Thanks
> -Abdelrhaman
>
>
> On Mar 7, 2013, at 10:33 AM, Sayan Kole <sa...@gmail.com> wrote:
>
> jobtracker in /hadoop/logs gets updated about the job status and
> everything with Log.info. But when I run it through eclipse, the log file
> of the jobtracker does not get written to. Is there somewhere in eclipse I
> have to specify something that I am missing?
>
>
>
Re: location of log files for mapreduce job run through eclipse
Posted by Sayan Kole <sa...@gmail.com>.
Ok. thanks. When I run the job from eclipse/command line, only certain
Log.info gets displayed on the console from certain files but not from
JobTracker. How do I display Log.info messages in JobTracker.java onto the
console like it does say for example Task.java shown below
13/03/07 12:31:17 INFO mapred.Task: Task 'attempt_local_0001_m_000000_0'
done.
13/03/07 12:31:17 INFO mapred.Task: Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@bdccedd
Thanks,
Sayan
On Thu, Mar 7, 2013 at 12:07 PM, Abdelrhman Shettia <
ashettia@hortonworks.com> wrote:
> Hi Sayan ,
>
> May want to try to use the log4j in hadoop and place it with the
> packages/jars and validate it matches the one in the cluster or print it to
> the console . I came across nice document that can help you in this case.
>
>
> http://www.thecloudavenue.com/2012/10/debugging-hadoop-mapreduce-program-in.html
>
>
> Thanks
> -Abdelrhaman
>
>
> On Mar 7, 2013, at 10:33 AM, Sayan Kole <sa...@gmail.com> wrote:
>
> jobtracker in /hadoop/logs gets updated about the job status and
> everything with Log.info. But when I run it through eclipse, the log file
> of the jobtracker does not get written to. Is there somewhere in eclipse I
> have to specify something that I am missing?
>
>
>
Re: location of log files for mapreduce job run through eclipse
Posted by Sayan Kole <sa...@gmail.com>.
Ok. thanks. When I run the job from eclipse/command line, only certain
Log.info gets displayed on the console from certain files but not from
JobTracker. How do I display Log.info messages in JobTracker.java onto the
console like it does say for example Task.java shown below
13/03/07 12:31:17 INFO mapred.Task: Task 'attempt_local_0001_m_000000_0'
done.
13/03/07 12:31:17 INFO mapred.Task: Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@bdccedd
Thanks,
Sayan
On Thu, Mar 7, 2013 at 12:07 PM, Abdelrhman Shettia <
ashettia@hortonworks.com> wrote:
> Hi Sayan ,
>
> May want to try to use the log4j in hadoop and place it with the
> packages/jars and validate it matches the one in the cluster or print it to
> the console . I came across nice document that can help you in this case.
>
>
> http://www.thecloudavenue.com/2012/10/debugging-hadoop-mapreduce-program-in.html
>
>
> Thanks
> -Abdelrhaman
>
>
> On Mar 7, 2013, at 10:33 AM, Sayan Kole <sa...@gmail.com> wrote:
>
> jobtracker in /hadoop/logs gets updated about the job status and
> everything with Log.info. But when I run it through eclipse, the log file
> of the jobtracker does not get written to. Is there somewhere in eclipse I
> have to specify something that I am missing?
>
>
>
Re: location of log files for mapreduce job run through eclipse
Posted by Sayan Kole <sa...@gmail.com>.
Ok. thanks. When I run the job from eclipse/command line, only certain
Log.info gets displayed on the console from certain files but not from
JobTracker. How do I display Log.info messages in JobTracker.java onto the
console like it does say for example Task.java shown below
13/03/07 12:31:17 INFO mapred.Task: Task 'attempt_local_0001_m_000000_0'
done.
13/03/07 12:31:17 INFO mapred.Task: Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@bdccedd
Thanks,
Sayan
On Thu, Mar 7, 2013 at 12:07 PM, Abdelrhman Shettia <
ashettia@hortonworks.com> wrote:
> Hi Sayan ,
>
> May want to try to use the log4j in hadoop and place it with the
> packages/jars and validate it matches the one in the cluster or print it to
> the console . I came across nice document that can help you in this case.
>
>
> http://www.thecloudavenue.com/2012/10/debugging-hadoop-mapreduce-program-in.html
>
>
> Thanks
> -Abdelrhaman
>
>
> On Mar 7, 2013, at 10:33 AM, Sayan Kole <sa...@gmail.com> wrote:
>
> jobtracker in /hadoop/logs gets updated about the job status and
> everything with Log.info. But when I run it through eclipse, the log file
> of the jobtracker does not get written to. Is there somewhere in eclipse I
> have to specify something that I am missing?
>
>
>
Re: location of log files for mapreduce job run through eclipse
Posted by Abdelrhman Shettia <as...@hortonworks.com>.
Hi Sayan ,
May want to try to use the log4j in hadoop and place it with the packages/jars and validate it matches the one in the cluster or print it to the console . I came across nice document that can help you in this case.
http://www.thecloudavenue.com/2012/10/debugging-hadoop-mapreduce-program-in.html
Thanks
-Abdelrhaman
On Mar 7, 2013, at 10:33 AM, Sayan Kole <sa...@gmail.com> wrote:
> jobtracker in /hadoop/logs gets updated about the job status and everything with Log.info. But when I run it through eclipse, the log file of the jobtracker does not get written to. Is there somewhere in eclipse I have to specify something that I am missing?
Re: location of log files for mapreduce job run through eclipse
Posted by Abdelrhman Shettia <as...@hortonworks.com>.
Hi Sayan ,
May want to try to use the log4j in hadoop and place it with the packages/jars and validate it matches the one in the cluster or print it to the console . I came across nice document that can help you in this case.
http://www.thecloudavenue.com/2012/10/debugging-hadoop-mapreduce-program-in.html
Thanks
-Abdelrhaman
On Mar 7, 2013, at 10:33 AM, Sayan Kole <sa...@gmail.com> wrote:
> jobtracker in /hadoop/logs gets updated about the job status and everything with Log.info. But when I run it through eclipse, the log file of the jobtracker does not get written to. Is there somewhere in eclipse I have to specify something that I am missing?
Re: location of log files for mapreduce job run through eclipse
Posted by Abdelrhman Shettia <as...@hortonworks.com>.
Hi Sayan ,
May want to try to use the log4j in hadoop and place it with the packages/jars and validate it matches the one in the cluster or print it to the console . I came across nice document that can help you in this case.
http://www.thecloudavenue.com/2012/10/debugging-hadoop-mapreduce-program-in.html
Thanks
-Abdelrhaman
On Mar 7, 2013, at 10:33 AM, Sayan Kole <sa...@gmail.com> wrote:
> jobtracker in /hadoop/logs gets updated about the job status and everything with Log.info. But when I run it through eclipse, the log file of the jobtracker does not get written to. Is there somewhere in eclipse I have to specify something that I am missing?
Re: location of log files for mapreduce job run through eclipse
Posted by Abdelrhman Shettia <as...@hortonworks.com>.
Hi Sayan ,
May want to try to use the log4j in hadoop and place it with the packages/jars and validate it matches the one in the cluster or print it to the console . I came across nice document that can help you in this case.
http://www.thecloudavenue.com/2012/10/debugging-hadoop-mapreduce-program-in.html
Thanks
-Abdelrhaman
On Mar 7, 2013, at 10:33 AM, Sayan Kole <sa...@gmail.com> wrote:
> jobtracker in /hadoop/logs gets updated about the job status and everything with Log.info. But when I run it through eclipse, the log file of the jobtracker does not get written to. Is there somewhere in eclipse I have to specify something that I am missing?
Re: location of log files for mapreduce job run through eclipse
Posted by Sayan Kole <sa...@gmail.com>.
When I run hadoop jobs through command line, then the log file of the
jobtracker in /hadoop/logs gets updated about the job status and everything
with Log.info. But when I run it through eclipse, the log file of the
jobtracker does not get written to. Is there somewhere in eclipse I have to
specify something that I am missing?
On Wed, Mar 6, 2013 at 6:01 PM, Harsh J <ha...@cloudera.com> wrote:
> If you've not changed any configs, look under /tmp/hadoop-${user.name}/
> perhaps.
>
> On Thu, Mar 7, 2013 at 3:19 AM, Sayan Kole <sa...@gmail.com> wrote:
> > Hi,
> > I cannot find the log files for the wordcount job: job_local_0001
> when I
> > run it through eclipse. I am getting the standard output on screen
> showing
> > me the progress of the maps/reduces, but when I check my log directory
> > inside the hadoop folder, I do not see any log files for this job nor
> does
> > the jobtracker log file have any information for this job.
> >
> > Sayan Kole
>
>
>
> --
> Harsh J
>
--
Sayan Kole,
Graduate Student,
School of Computing, Informatics, and Decision Systems Engineering,
Arizona State University,
Re: location of log files for mapreduce job run through eclipse
Posted by Sayan Kole <sa...@gmail.com>.
When I run hadoop jobs through command line, then the log file of the
jobtracker in /hadoop/logs gets updated about the job status and everything
with Log.info. But when I run it through eclipse, the log file of the
jobtracker does not get written to. Is there somewhere in eclipse I have to
specify something that I am missing?
On Wed, Mar 6, 2013 at 6:01 PM, Harsh J <ha...@cloudera.com> wrote:
> If you've not changed any configs, look under /tmp/hadoop-${user.name}/
> perhaps.
>
> On Thu, Mar 7, 2013 at 3:19 AM, Sayan Kole <sa...@gmail.com> wrote:
> > Hi,
> > I cannot find the log files for the wordcount job: job_local_0001
> when I
> > run it through eclipse. I am getting the standard output on screen
> showing
> > me the progress of the maps/reduces, but when I check my log directory
> > inside the hadoop folder, I do not see any log files for this job nor
> does
> > the jobtracker log file have any information for this job.
> >
> > Sayan Kole
>
>
>
> --
> Harsh J
>
--
Sayan Kole,
Graduate Student,
School of Computing, Informatics, and Decision Systems Engineering,
Arizona State University,
Re: location of log files for mapreduce job run through eclipse
Posted by Sayan Kole <sa...@gmail.com>.
When I run hadoop jobs through command line, then the log file of the
jobtracker in /hadoop/logs gets updated about the job status and everything
with Log.info. But when I run it through eclipse, the log file of the
jobtracker does not get written to. Is there somewhere in eclipse I have to
specify something that I am missing?
On Wed, Mar 6, 2013 at 6:01 PM, Harsh J <ha...@cloudera.com> wrote:
> If you've not changed any configs, look under /tmp/hadoop-${user.name}/
> perhaps.
>
> On Thu, Mar 7, 2013 at 3:19 AM, Sayan Kole <sa...@gmail.com> wrote:
> > Hi,
> > I cannot find the log files for the wordcount job: job_local_0001
> when I
> > run it through eclipse. I am getting the standard output on screen
> showing
> > me the progress of the maps/reduces, but when I check my log directory
> > inside the hadoop folder, I do not see any log files for this job nor
> does
> > the jobtracker log file have any information for this job.
> >
> > Sayan Kole
>
>
>
> --
> Harsh J
>
--
Sayan Kole,
Graduate Student,
School of Computing, Informatics, and Decision Systems Engineering,
Arizona State University,
Re: location of log files for mapreduce job run through eclipse
Posted by Sayan Kole <sa...@gmail.com>.
When I run hadoop jobs through command line, then the log file of the
jobtracker in /hadoop/logs gets updated about the job status and everything
with Log.info. But when I run it through eclipse, the log file of the
jobtracker does not get written to. Is there somewhere in eclipse I have to
specify something that I am missing?
On Wed, Mar 6, 2013 at 6:01 PM, Harsh J <ha...@cloudera.com> wrote:
> If you've not changed any configs, look under /tmp/hadoop-${user.name}/
> perhaps.
>
> On Thu, Mar 7, 2013 at 3:19 AM, Sayan Kole <sa...@gmail.com> wrote:
> > Hi,
> > I cannot find the log files for the wordcount job: job_local_0001
> when I
> > run it through eclipse. I am getting the standard output on screen
> showing
> > me the progress of the maps/reduces, but when I check my log directory
> > inside the hadoop folder, I do not see any log files for this job nor
> does
> > the jobtracker log file have any information for this job.
> >
> > Sayan Kole
>
>
>
> --
> Harsh J
>
--
Sayan Kole,
Graduate Student,
School of Computing, Informatics, and Decision Systems Engineering,
Arizona State University,
Re: location of log files for mapreduce job run through eclipse
Posted by Harsh J <ha...@cloudera.com>.
If you've not changed any configs, look under /tmp/hadoop-${user.name}/ perhaps.
On Thu, Mar 7, 2013 at 3:19 AM, Sayan Kole <sa...@gmail.com> wrote:
> Hi,
> I cannot find the log files for the wordcount job: job_local_0001 when I
> run it through eclipse. I am getting the standard output on screen showing
> me the progress of the maps/reduces, but when I check my log directory
> inside the hadoop folder, I do not see any log files for this job nor does
> the jobtracker log file have any information for this job.
>
> Sayan Kole
--
Harsh J
Re: location of log files for mapreduce job run through eclipse
Posted by Harsh J <ha...@cloudera.com>.
If you've not changed any configs, look under /tmp/hadoop-${user.name}/ perhaps.
On Thu, Mar 7, 2013 at 3:19 AM, Sayan Kole <sa...@gmail.com> wrote:
> Hi,
> I cannot find the log files for the wordcount job: job_local_0001 when I
> run it through eclipse. I am getting the standard output on screen showing
> me the progress of the maps/reduces, but when I check my log directory
> inside the hadoop folder, I do not see any log files for this job nor does
> the jobtracker log file have any information for this job.
>
> Sayan Kole
--
Harsh J
Re: location of log files for mapreduce job run through eclipse
Posted by Harsh J <ha...@cloudera.com>.
If you've not changed any configs, look under /tmp/hadoop-${user.name}/ perhaps.
On Thu, Mar 7, 2013 at 3:19 AM, Sayan Kole <sa...@gmail.com> wrote:
> Hi,
> I cannot find the log files for the wordcount job: job_local_0001 when I
> run it through eclipse. I am getting the standard output on screen showing
> me the progress of the maps/reduces, but when I check my log directory
> inside the hadoop folder, I do not see any log files for this job nor does
> the jobtracker log file have any information for this job.
>
> Sayan Kole
--
Harsh J
Re: location of log files for mapreduce job run through eclipse
Posted by Harsh J <ha...@cloudera.com>.
If you've not changed any configs, look under /tmp/hadoop-${user.name}/ perhaps.
On Thu, Mar 7, 2013 at 3:19 AM, Sayan Kole <sa...@gmail.com> wrote:
> Hi,
> I cannot find the log files for the wordcount job: job_local_0001 when I
> run it through eclipse. I am getting the standard output on screen showing
> me the progress of the maps/reduces, but when I check my log directory
> inside the hadoop folder, I do not see any log files for this job nor does
> the jobtracker log file have any information for this job.
>
> Sayan Kole
--
Harsh J