You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by "W.P. McNeill" <bi...@gmail.com> on 2011/10/04 23:22:06 UTC
Re: How do I diagnose IO bounded errors using the framework counters?
Here's an even more basic question. I tried to figure out what
the FILE_BYTES_READ means by searching every file in the hadoop 0.20.203.0
installation for the string FILE_BYTES_READ installation by running
find . -type f | xargs grep FILE_BYTES_READ
I only found this string in source files in vaidya contributor directory and
the tools/rumen directories. Nothing in the main source base.
Where in the source code are these counters created and updated?
Re: How do I diagnose IO bounded errors using the framework counters?
Posted by John Meagher <jo...@gmail.com>.
The counter names are created dynamically in mapred.Task
/**
* Counters to measure the usage of the different file systems.
* Always return the String array with two elements. First one is
the name of
* BYTES_READ counter and second one is of the BYTES_WRITTEN counter.
*/
protected static String[] getFileSystemCounterNames(String uriScheme) {
String scheme = uriScheme.toUpperCase();
return new String[]{scheme+"_BYTES_READ", scheme+"_BYTES_WRITTEN"};
}
On Tue, Oct 4, 2011 at 17:22, W.P. McNeill <bi...@gmail.com> wrote:
> Here's an even more basic question. I tried to figure out what
> the FILE_BYTES_READ means by searching every file in the hadoop 0.20.203.0
> installation for the string FILE_BYTES_READ installation by running
>
> find . -type f | xargs grep FILE_BYTES_READ
>
> I only found this string in source files in vaidya contributor directory and
> the tools/rumen directories. Nothing in the main source base.
>
> Where in the source code are these counters created and updated?
>