Tez Analyzer Usage Question

[Jonathan Eagles <je...@gmail.com>]

Can the job analyzer be used at this point for the eager developer? I would
like to get started using the functionality that is there. I have ATS data
for jobs and would be able to analyzer slow jobs, but don't have
instructions on what can be done at this point.

I'm assuming there is a way to do this by downloading the ATS data, parsing
the ats data, and pointing the current set of analyzers at it. Just needing
a push in the right direction I think.

Jon

Re: Tez Analyzer Usage Question

[Rajesh Balamohan <rb...@apache.org>]

There is no rendering mechanism OOTB for displaying the job analyzer
results. Tool base jar which Bikas mentioned would definitely simplify the
way the analyzers are run and print the results on to console. Very basic
(PoC mode) standalone UI patch which uses Vaadin web framework is uploaded
in TEZ-2729. Please note that this is very very basic implementation which
lets user to run through a bunch of analyzers and render the results in
separate tabs.

Alternatively, tools like Zeppelin can be used for displaying the results
of job-analyzers. Advantage is that, it allows data to be stored in RDD
which can be linked to DataFrame and queried at runtime. It supports basic
charting as well. E.g notebook implementation is provided in
https://github.com/rajeshbalamohan/zeppelin-jobAnalyzer-demo-ATSData
(needless to say, this notebook is like PoC mode. :)  ).

~Rajesh.B


On Tue, Aug 18, 2015 at 3:48 AM, Jonathan Eagles <je...@gmail.com> wrote:

> Looking forward to it. Do you see this being tied to a tez cli as in
> TEZ-2658 <https://issues.apache.org/jira/browse/TEZ-2658> or as a stand
> alone?
> Jon
> <https://issues.apache.org/jira/browse/TEZ-2658>
>
> On Mon, Aug 17, 2015 at 5:12 PM, Bikas Saha <bi...@hortonworks.com> wrote:
>
> > I believe so :) with the eager developer caveats. I am working on a Tool
> > based jar so that the analysers could be run from the command line using
> > "Hadoop jar" instead of having to write code to invoke them. Please give
> me
> > a day or two for that.
> >
> > Bikas
> >
> > -----Original Message-----
> > From: Jonathan Eagles [mailto:jeagles@gmail.com]
> > Sent: Monday, August 17, 2015 3:01 PM
> > To: dev@tez.apache.org
> > Subject: Tez Analyzer Usage Question
> >
> > Can the job analyzer be used at this point for the eager developer? I
> > would like to get started using the functionality that is there. I have
> ATS
> > data for jobs and would be able to analyzer slow jobs, but don't have
> > instructions on what can be done at this point.
> >
> > I'm assuming there is a way to do this by downloading the ATS data,
> > parsing the ats data, and pointing the current set of analyzers at it.
> Just
> > needing a push in the right direction I think.
> >
> > Jon
> >
>

Re: Tez Analyzer Usage Question

[Jonathan Eagles <je...@gmail.com>]

Looking forward to it. Do you see this being tied to a tez cli as in
TEZ-2658 <https://issues.apache.org/jira/browse/TEZ-2658> or as a stand
alone?
Jon
<https://issues.apache.org/jira/browse/TEZ-2658>

On Mon, Aug 17, 2015 at 5:12 PM, Bikas Saha <bi...@hortonworks.com> wrote:

> I believe so :) with the eager developer caveats. I am working on a Tool
> based jar so that the analysers could be run from the command line using
> "Hadoop jar" instead of having to write code to invoke them. Please give me
> a day or two for that.
>
> Bikas
>
> -----Original Message-----
> From: Jonathan Eagles [mailto:jeagles@gmail.com]
> Sent: Monday, August 17, 2015 3:01 PM
> To: dev@tez.apache.org
> Subject: Tez Analyzer Usage Question
>
> Can the job analyzer be used at this point for the eager developer? I
> would like to get started using the functionality that is there. I have ATS
> data for jobs and would be able to analyzer slow jobs, but don't have
> instructions on what can be done at this point.
>
> I'm assuming there is a way to do this by downloading the ATS data,
> parsing the ats data, and pointing the current set of analyzers at it. Just
> needing a push in the right direction I think.
>
> Jon
>

RE: Tez Analyzer Usage Question

[Bikas Saha <bi...@hortonworks.com>]

I believe so :) with the eager developer caveats. I am working on a Tool based jar so that the analysers could be run from the command line using "Hadoop jar" instead of having to write code to invoke them. Please give me a day or two for that.

Bikas

-----Original Message-----
From: Jonathan Eagles [mailto:jeagles@gmail.com] 
Sent: Monday, August 17, 2015 3:01 PM
To: dev@tez.apache.org
Subject: Tez Analyzer Usage Question

Can the job analyzer be used at this point for the eager developer? I would like to get started using the functionality that is there. I have ATS data for jobs and would be able to analyzer slow jobs, but don't have instructions on what can be done at this point.

I'm assuming there is a way to do this by downloading the ATS data, parsing the ats data, and pointing the current set of analyzers at it. Just needing a push in the right direction I think.

Jon