You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Ryan Rosario <uc...@gmail.com> on 2011/11/14 19:51:25 UTC

Reading Global Counters in Streaming?

Hi,

I am writing a Hadoop Streaming job in Python. I know that I can
increment counters by writing a special format to sys.stderr. Is it
possible to *read* the values of counters from my Python program? I am
using the global counter as the denominator of a probability, and must
have this value available to all reducers. Is this possible, and how
would I do it in Python?

Thanks,
Ryan