You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Zhenhua Guo <je...@gmail.com> on 2011/01/22 18:57:47 UTC
How to get metrics information?
I want to get metrics information (e.g. number of Maps, number of
Reduces, memory use, load) by APIs. I found two useful classes -
ClusterStatus and ClusterMetrics. My question is how I can get
instances of that two classes? From JobClient or JobTracker? Any
suggested alternative way to get the information?
Thanks
Gerald
Re: How to get metrics information?
Posted by Edward Capriolo <ed...@gmail.com>.
On Sat, Jan 22, 2011 at 9:59 PM, Ted Yu <yu...@gmail.com> wrote:
> In the test code, JobTracker is returned from:
>
> mr = new MiniMRCluster(0, 0, 0, "file:///", 1, null, null, null,
> conf);
> jobTracker = mr.getJobTrackerRunner().getJobTracker();
>
> I guess it is not exposed in non-test code.
>
> On Sat, Jan 22, 2011 at 6:38 PM, Zhenhua Guo <je...@gmail.com> wrote:
>
>> Thanks!
>> How to get JobTracker object?
>>
>> Gerald
>>
>> On Sun, Jan 23, 2011 at 5:46 AM, Ted Yu <yu...@gmail.com> wrote:
>> > You can use the following code:
>> > JobClient jc = new JobClient(jobConf);
>> > int numReduces = jc.getClusterStatus().getMaxReduceTasks();
>> >
>> > For 0.20.3, you can use:
>> > ClusterMetrics metrics = jobTracker.getClusterMetrics();
>> >
>> > On Sat, Jan 22, 2011 at 9:57 AM, Zhenhua Guo <je...@gmail.com> wrote:
>> >
>> >> I want to get metrics information (e.g. number of Maps, number of
>> >> Reduces, memory use, load) by APIs. I found two useful classes -
>> >> ClusterStatus and ClusterMetrics. My question is how I can get
>> >> instances of that two classes? From JobClient or JobTracker? Any
>> >> suggested alternative way to get the information?
>> >>
>> >> Thanks
>> >>
>> >> Gerald
>> >>
>> >
>>
>
Correct JobTracker is the JobTracker itself that does not have any interface.
However using JobClient you can mine most of the information out from
the Job Tracker.
My cacti graphing package takes that exact approach to pull
information that is not a JMXCounter such as.
http://www.jointhegrid.com/hadoop-cacti-jtg-walk/maps_v_reduces.jsp
Re: How to get metrics information?
Posted by Ted Yu <yu...@gmail.com>.
In the test code, JobTracker is returned from:
mr = new MiniMRCluster(0, 0, 0, "file:///", 1, null, null, null,
conf);
jobTracker = mr.getJobTrackerRunner().getJobTracker();
I guess it is not exposed in non-test code.
On Sat, Jan 22, 2011 at 6:38 PM, Zhenhua Guo <je...@gmail.com> wrote:
> Thanks!
> How to get JobTracker object?
>
> Gerald
>
> On Sun, Jan 23, 2011 at 5:46 AM, Ted Yu <yu...@gmail.com> wrote:
> > You can use the following code:
> > JobClient jc = new JobClient(jobConf);
> > int numReduces = jc.getClusterStatus().getMaxReduceTasks();
> >
> > For 0.20.3, you can use:
> > ClusterMetrics metrics = jobTracker.getClusterMetrics();
> >
> > On Sat, Jan 22, 2011 at 9:57 AM, Zhenhua Guo <je...@gmail.com> wrote:
> >
> >> I want to get metrics information (e.g. number of Maps, number of
> >> Reduces, memory use, load) by APIs. I found two useful classes -
> >> ClusterStatus and ClusterMetrics. My question is how I can get
> >> instances of that two classes? From JobClient or JobTracker? Any
> >> suggested alternative way to get the information?
> >>
> >> Thanks
> >>
> >> Gerald
> >>
> >
>
Re: How to get metrics information?
Posted by Zhenhua Guo <je...@gmail.com>.
Thanks!
How to get JobTracker object?
Gerald
On Sun, Jan 23, 2011 at 5:46 AM, Ted Yu <yu...@gmail.com> wrote:
> You can use the following code:
> JobClient jc = new JobClient(jobConf);
> int numReduces = jc.getClusterStatus().getMaxReduceTasks();
>
> For 0.20.3, you can use:
> ClusterMetrics metrics = jobTracker.getClusterMetrics();
>
> On Sat, Jan 22, 2011 at 9:57 AM, Zhenhua Guo <je...@gmail.com> wrote:
>
>> I want to get metrics information (e.g. number of Maps, number of
>> Reduces, memory use, load) by APIs. I found two useful classes -
>> ClusterStatus and ClusterMetrics. My question is how I can get
>> instances of that two classes? From JobClient or JobTracker? Any
>> suggested alternative way to get the information?
>>
>> Thanks
>>
>> Gerald
>>
>
Re: How to get metrics information?
Posted by Ted Yu <yu...@gmail.com>.
You can use the following code:
JobClient jc = new JobClient(jobConf);
int numReduces = jc.getClusterStatus().getMaxReduceTasks();
For 0.20.3, you can use:
ClusterMetrics metrics = jobTracker.getClusterMetrics();
On Sat, Jan 22, 2011 at 9:57 AM, Zhenhua Guo <je...@gmail.com> wrote:
> I want to get metrics information (e.g. number of Maps, number of
> Reduces, memory use, load) by APIs. I found two useful classes -
> ClusterStatus and ClusterMetrics. My question is how I can get
> instances of that two classes? From JobClient or JobTracker? Any
> suggested alternative way to get the information?
>
> Thanks
>
> Gerald
>