You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by ArunKumar <ar...@gmail.com> on 2011/12/14 15:40:34 UTC
Analysing Completed Job info programmatically apart from Jobtracker
GUI
Hi Guys !
I want to analyse the completed Job counters like FILE/HDFS BYTES
READ/WRITTEN along with other values like average map/reduce task run time.
I see that Jobtracker GUI has this info but i want to programmatically
retrieve these values instead of manually noting down these values and do
some analysis. Can i do it in a simple/easier way ?
I also see that Cloudera's HUE is good for this but is there anything
equivalent in Hadoop.
Can anyone guide me in this regard ?
Arun
--
View this message in context: http://lucene.472066.n3.nabble.com/Analysing-Completed-Job-info-programmatically-apart-from-Jobtracker-GUI-tp3585629p3585629.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Re: Analysing Completed Job info programmatically apart from
Jobtracker GUI
Posted by ArunKumar <ar...@gmail.com>.
Edward,
I have checked it out.
Can u provide me the installations guidelines and prerequisites(like Hadoop
version, Cassandra,..) for using this ?
I don't see any info regarding run or build. Can u tell me how to run in it
in detail and in what order (say hadoop daemons,etc) ?
Thanks,
Arun
On Wed, Dec 14, 2011 at 11:01 PM, Edward Capriolo [via Lucene] <
ml-node+s472066n3586151h60@n3.nabble.com> wrote:
> I would check out hitune. I have a github project that connects to the
> JobTracker and stores counters, job times and other stats into Cassandra.
>
> https://github.com/edwardcapriolo/hadoop_cluster_profiler
>
> Worth checking out as discovering how to connect and mine information from
> the JobTracker was quite fun.
>
> Edward
>
>
>
> On Wed, Dec 14, 2011 at 9:40 AM, ArunKumar <[hidden email]<http://user/SendEmail.jtp?type=node&node=3586151&i=0>>
> wrote:
>
> > Hi Guys !
> >
> > I want to analyse the completed Job counters like FILE/HDFS BYTES
> > READ/WRITTEN along with other values like average map/reduce task run
> time.
> > I see that Jobtracker GUI has this info but i want to programmatically
> > retrieve these values instead of manually noting down these values and
> do
> > some analysis. Can i do it in a simple/easier way ?
> > I also see that Cloudera's HUE is good for this but is there anything
> > equivalent in Hadoop.
> >
> > Can anyone guide me in this regard ?
> >
> > Arun
> >
> > --
> > View this message in context:
> >
> http://lucene.472066.n3.nabble.com/Analysing-Completed-Job-info-programmatically-apart-from-Jobtracker-GUI-tp3585629p3585629.html
> > Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
> >
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/Analysing-Completed-Job-info-programmatically-apart-from-Jobtracker-GUI-tp3585629p3586151.html
> To unsubscribe from Analysing Completed Job info programmatically apart
> from Jobtracker GUI, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3585629&code=YXJ1bms3ODZAZ21haWwuY29tfDM1ODU2Mjl8NzA5NTc4MTY3>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.InstantMailNamespace&breadcrumbs=instant+emails%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/Analysing-Completed-Job-info-programmatically-apart-from-Jobtracker-GUI-tp3585629p3587378.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Re: Analysing Completed Job info programmatically apart from
Jobtracker GUI
Posted by Edward Capriolo <ed...@gmail.com>.
I would check out hitune. I have a github project that connects to the
JobTracker and stores counters, job times and other stats into Cassandra.
https://github.com/edwardcapriolo/hadoop_cluster_profiler
Worth checking out as discovering how to connect and mine information from
the JobTracker was quite fun.
Edward
On Wed, Dec 14, 2011 at 9:40 AM, ArunKumar <ar...@gmail.com> wrote:
> Hi Guys !
>
> I want to analyse the completed Job counters like FILE/HDFS BYTES
> READ/WRITTEN along with other values like average map/reduce task run time.
> I see that Jobtracker GUI has this info but i want to programmatically
> retrieve these values instead of manually noting down these values and do
> some analysis. Can i do it in a simple/easier way ?
> I also see that Cloudera's HUE is good for this but is there anything
> equivalent in Hadoop.
>
> Can anyone guide me in this regard ?
>
> Arun
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Analysing-Completed-Job-info-programmatically-apart-from-Jobtracker-GUI-tp3585629p3585629.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
>