Task Counters

[Kha <su...@gmail.com>]

Hi all,

I understand that task counters are updated by tasks, and they can be seen
by the AM only. The counters information go in one direction (Tasks --> AM)

My question: Is there any way where I can let tasks see some task counters
which are updated by the AM (Tasks <--> AM)?

Thanks,

Re: Task Counters

[Vinod Kumar Vavilapalli <vi...@apache.org>]

When you say "counters updated by AM", do you mean
 (a) the Job Counters or
 (b) counters written by a custom AM that extends MR AM?

If it's (a), the only way is for the Task to use JobClient API to get those counters. For this to work in secure clusters, you will have to get YARN RM delegation-tokens and put them into the Job.

HTH

+Vinod

> On May 5, 2019, at 5:16 PM, Kha <su...@gmail.com> wrote:
> 
> Hi all,
> 
> I understand that task counters are updated by tasks, and they can be seen
> by the AM only. The counters information go in one direction (Tasks --> AM)
> 
> My question: Is there any way where I can let tasks see some task counters
> which are updated by the AM (Tasks <--> AM)?
> 
> Thanks,


---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org