You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tez.apache.org by Premal Shah <pr...@gmail.com> on 2016/11/09 16:48:32 UTC
Issue with the job progress in the UI
Hi.
I'm running a hive on tez query and this is what shows up in the shell
Logging initialized using configuration in
file:/usr/lib/apache-hive-2.0.1-bin/conf/hive-log4j2.properties
OK
Time taken: 2.422 seconds
OK
Time taken: 0.089 seconds
Query ID = hadoop_20161109162801_2432d154-c75b-4173-af86-257a00b849f8
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1478236559237_1263)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING
PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 48 48 0
0 0 0
Map 10 ......... container SUCCEEDED 7 7 0
0 0 0
Map 11 ......... container SUCCEEDED 14 14 0
0 0 0
Map 7 .......... container SUCCEEDED 44 44 0
0 0 0
Map 8 .......... container SUCCEEDED 12 12 0
0 0 0
Map 9 .......... container SUCCEEDED 44 44 0
0 0 0
Reducer 3 ..... container RUNNING 32 31 1
0 0 0
Reducer 4 container RUNNING 27 0 27
0 0 0
Reducer 5 container INITED 31 0 0
31 0 0
Reducer 6 container INITED 32 0 0
32 0 0
----------------------------------------------------------------------------------------------
VERTICES: 06/10 [=================>>---------] 68% ELAPSED TIME: 893.31
s
----------------------------------------------------------------------------------------------
The query is still running.
When I head over to the UI, it does not show the reducers
[image: Inline image 1]
And this does not happen for every query. Is there a limit to number of
rows in the UI table?
--
Regards,
Premal Shah.
Re: Issue with the job progress in the UI
Posted by Premal Shah <pr...@gmail.com>.
Created - https://issues.apache.org/jira/browse/TEZ-3538
On Wed, Nov 9, 2016 at 9:54 AM, Premal Shah <pr...@gmail.com> wrote:
> Thanx Hitesh.
> Got it. Will look for the JIRA or create a new one.
>
> On Wed, Nov 9, 2016 at 9:21 AM, Hitesh Shah <hi...@apache.org> wrote:
>
>> Hello Premal,
>>
>> This is likely a combination of a lag in publishing the history events to
>> YARN timeline which is consumed by the UI and also related to the UI
>> relying more on YARN Timeline for data as compared to reading the
>> information directly from the Tez AM. The Hive client is directly getting
>> its info from the AM but the UI is using a mix of both leading to such
>> confusion.
>>
>> I believe there might already be an open jira for this but if you don’t
>> mind, can you go ahead and create a new one in any case with the details
>> that you have noticed?
>>
>> thanks
>> — Hitesh
>>
>>
>> > On Nov 9, 2016, at 8:48 AM, Premal Shah <pr...@gmail.com>
>> wrote:
>> >
>> > Hi.
>> > I'm running a hive on tez query and this is what shows up in the shell
>> >
>> > Logging initialized using configuration in
>> file:/usr/lib/apache-hive-2.0.1-bin/conf/hive-log4j2.properties
>> > OK
>> > Time taken: 2.422 seconds
>> > OK
>> > Time taken: 0.089 seconds
>> > Query ID = hadoop_20161109162801_2432d154-c75b-4173-af86-257a00b849f8
>> > Total jobs = 1
>> > Launching Job 1 out of 1
>> >
>> >
>> > Status: Running (Executing on YARN cluster with App id
>> application_1478236559237_1263)
>> >
>> > ------------------------------------------------------------
>> ----------------------------------
>> > VERTICES MODE STATUS TOTAL COMPLETED RUNNING
>> PENDING FAILED KILLED
>> > ------------------------------------------------------------
>> ----------------------------------
>> > Map 1 .......... container SUCCEEDED 48 48 0
>> 0 0 0
>> > Map 10 ......... container SUCCEEDED 7 7 0
>> 0 0 0
>> > Map 11 ......... container SUCCEEDED 14 14 0
>> 0 0 0
>> > Map 7 .......... container SUCCEEDED 44 44 0
>> 0 0 0
>> > Map 8 .......... container SUCCEEDED 12 12 0
>> 0 0 0
>> > Map 9 .......... container SUCCEEDED 44 44 0
>> 0 0 0
>> > Reducer 3 ..... container RUNNING 32 31 1
>> 0 0 0
>> > Reducer 4 container RUNNING 27 0 27
>> 0 0 0
>> > Reducer 5 container INITED 31 0 0
>> 31 0 0
>> > Reducer 6 container INITED 32 0 0
>> 32 0 0
>> > ------------------------------------------------------------
>> ----------------------------------
>> > VERTICES: 06/10 [=================>>---------] 68% ELAPSED TIME:
>> 893.31 s
>> > ------------------------------------------------------------
>> ----------------------------------
>> >
>> > The query is still running.
>> >
>> > When I head over to the UI, it does not show the reducers
>> >
>> > <Screenshot 2016-11-09 08.42.42.png>
>> >
>> >
>> > And this does not happen for every query. Is there a limit to number of
>> rows in the UI table?
>> >
>> >
>> > --
>> > Regards,
>> > Premal Shah.
>>
>>
>
>
> --
> Regards,
> Premal Shah.
>
--
Regards,
Premal Shah.
Re: Issue with the job progress in the UI
Posted by Premal Shah <pr...@gmail.com>.
Thanx Hitesh.
Got it. Will look for the JIRA or create a new one.
On Wed, Nov 9, 2016 at 9:21 AM, Hitesh Shah <hi...@apache.org> wrote:
> Hello Premal,
>
> This is likely a combination of a lag in publishing the history events to
> YARN timeline which is consumed by the UI and also related to the UI
> relying more on YARN Timeline for data as compared to reading the
> information directly from the Tez AM. The Hive client is directly getting
> its info from the AM but the UI is using a mix of both leading to such
> confusion.
>
> I believe there might already be an open jira for this but if you don’t
> mind, can you go ahead and create a new one in any case with the details
> that you have noticed?
>
> thanks
> — Hitesh
>
>
> > On Nov 9, 2016, at 8:48 AM, Premal Shah <pr...@gmail.com> wrote:
> >
> > Hi.
> > I'm running a hive on tez query and this is what shows up in the shell
> >
> > Logging initialized using configuration in file:/usr/lib/apache-hive-2.0.
> 1-bin/conf/hive-log4j2.properties
> > OK
> > Time taken: 2.422 seconds
> > OK
> > Time taken: 0.089 seconds
> > Query ID = hadoop_20161109162801_2432d154-c75b-4173-af86-257a00b849f8
> > Total jobs = 1
> > Launching Job 1 out of 1
> >
> >
> > Status: Running (Executing on YARN cluster with App id
> application_1478236559237_1263)
> >
> > ------------------------------------------------------------
> ----------------------------------
> > VERTICES MODE STATUS TOTAL COMPLETED RUNNING
> PENDING FAILED KILLED
> > ------------------------------------------------------------
> ----------------------------------
> > Map 1 .......... container SUCCEEDED 48 48 0
> 0 0 0
> > Map 10 ......... container SUCCEEDED 7 7 0
> 0 0 0
> > Map 11 ......... container SUCCEEDED 14 14 0
> 0 0 0
> > Map 7 .......... container SUCCEEDED 44 44 0
> 0 0 0
> > Map 8 .......... container SUCCEEDED 12 12 0
> 0 0 0
> > Map 9 .......... container SUCCEEDED 44 44 0
> 0 0 0
> > Reducer 3 ..... container RUNNING 32 31 1
> 0 0 0
> > Reducer 4 container RUNNING 27 0 27
> 0 0 0
> > Reducer 5 container INITED 31 0 0
> 31 0 0
> > Reducer 6 container INITED 32 0 0
> 32 0 0
> > ------------------------------------------------------------
> ----------------------------------
> > VERTICES: 06/10 [=================>>---------] 68% ELAPSED TIME:
> 893.31 s
> > ------------------------------------------------------------
> ----------------------------------
> >
> > The query is still running.
> >
> > When I head over to the UI, it does not show the reducers
> >
> > <Screenshot 2016-11-09 08.42.42.png>
> >
> >
> > And this does not happen for every query. Is there a limit to number of
> rows in the UI table?
> >
> >
> > --
> > Regards,
> > Premal Shah.
>
>
--
Regards,
Premal Shah.
Re: Issue with the job progress in the UI
Posted by Hitesh Shah <hi...@apache.org>.
Hello Premal,
This is likely a combination of a lag in publishing the history events to YARN timeline which is consumed by the UI and also related to the UI relying more on YARN Timeline for data as compared to reading the information directly from the Tez AM. The Hive client is directly getting its info from the AM but the UI is using a mix of both leading to such confusion.
I believe there might already be an open jira for this but if you don’t mind, can you go ahead and create a new one in any case with the details that you have noticed?
thanks
— Hitesh
> On Nov 9, 2016, at 8:48 AM, Premal Shah <pr...@gmail.com> wrote:
>
> Hi.
> I'm running a hive on tez query and this is what shows up in the shell
>
> Logging initialized using configuration in file:/usr/lib/apache-hive-2.0.1-bin/conf/hive-log4j2.properties
> OK
> Time taken: 2.422 seconds
> OK
> Time taken: 0.089 seconds
> Query ID = hadoop_20161109162801_2432d154-c75b-4173-af86-257a00b849f8
> Total jobs = 1
> Launching Job 1 out of 1
>
>
> Status: Running (Executing on YARN cluster with App id application_1478236559237_1263)
>
> ----------------------------------------------------------------------------------------------
> VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
> ----------------------------------------------------------------------------------------------
> Map 1 .......... container SUCCEEDED 48 48 0 0 0 0
> Map 10 ......... container SUCCEEDED 7 7 0 0 0 0
> Map 11 ......... container SUCCEEDED 14 14 0 0 0 0
> Map 7 .......... container SUCCEEDED 44 44 0 0 0 0
> Map 8 .......... container SUCCEEDED 12 12 0 0 0 0
> Map 9 .......... container SUCCEEDED 44 44 0 0 0 0
> Reducer 3 ..... container RUNNING 32 31 1 0 0 0
> Reducer 4 container RUNNING 27 0 27 0 0 0
> Reducer 5 container INITED 31 0 0 31 0 0
> Reducer 6 container INITED 32 0 0 32 0 0
> ----------------------------------------------------------------------------------------------
> VERTICES: 06/10 [=================>>---------] 68% ELAPSED TIME: 893.31 s
> ----------------------------------------------------------------------------------------------
>
> The query is still running.
>
> When I head over to the UI, it does not show the reducers
>
> <Screenshot 2016-11-09 08.42.42.png>
>
>
> And this does not happen for every query. Is there a limit to number of rows in the UI table?
>
>
> --
> Regards,
> Premal Shah.