You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Philip Zeyliger (JIRA)" <ji...@apache.org> on 2010/08/30 20:09:53 UTC
[jira] Created: (MAPREDUCE-2043) TaskTrackerInstrumentation and
JobTrackerInstrumentation should be public
TaskTrackerInstrumentation and JobTrackerInstrumentation should be public
-------------------------------------------------------------------------
Key: MAPREDUCE-2043
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2043
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: tasktracker
Affects Versions: 0.22.0
Reporter: Philip Zeyliger
Assignee: Philip Zeyliger
Attachments: MAPREDUCE-2043.patch.txt
Hadoop administrators can specify classes to be loaded as "TaskTrackerInstrumentation" and "JobTrackerInstrumentation" implementations, which, roughly, define listeners on TT and JT events. Unfortunately, since the class has default access, extending it requires setting the extension's package to org.apache.hadoop.mapred, which seems like poor form.
I propose we make the two instrumentation classes public, so they can be extended wherever.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2043) TaskTrackerInstrumentation and
JobTrackerInstrumentation should be public
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904296#action_12904296 ]
Chris Douglas commented on MAPREDUCE-2043:
------------------------------------------
Most of the listeners it defines are called during heartbeats... allowing administrators to put arbitrary code in that inner loop was not in its design. Have you looked at HADOOP-6728? If I understand the design doc, it allows a {{MetricsSink}} to be defined and notified outside of that cycle.
> TaskTrackerInstrumentation and JobTrackerInstrumentation should be public
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-2043
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2043
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.22.0
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Attachments: MAPREDUCE-2043.patch.txt
>
>
> Hadoop administrators can specify classes to be loaded as "TaskTrackerInstrumentation" and "JobTrackerInstrumentation" implementations, which, roughly, define listeners on TT and JT events. Unfortunately, since the class has default access, extending it requires setting the extension's package to org.apache.hadoop.mapred, which seems like poor form.
> I propose we make the two instrumentation classes public, so they can be extended wherever.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2043) TaskTrackerInstrumentation and
JobTrackerInstrumentation should be public
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904476#action_12904476 ]
Luke Lu commented on MAPREDUCE-2043:
------------------------------------
@Chris, see MAPREDUCE-1881 for a summary comparison between the instrumentation interface and the metrics framework.
The intended audience for the instrumentation classes is NOT administrators but experimental framework developers that either embed or extend Hadoop.
-1 for making the interface public in the sense that it's our (Hadoop community) obligation to maintain/document the interface. I'd strongly advocate that we reserve the right to change instrumentation interface without the usual constraints of public APIs.
+0 if proper interface classification annotations are used i.e.:
@InterfaceAudience.Private
@InterfaceStability.Evolving
> TaskTrackerInstrumentation and JobTrackerInstrumentation should be public
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-2043
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2043
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.22.0
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Attachments: MAPREDUCE-2043.patch.txt
>
>
> Hadoop administrators can specify classes to be loaded as "TaskTrackerInstrumentation" and "JobTrackerInstrumentation" implementations, which, roughly, define listeners on TT and JT events. Unfortunately, since the class has default access, extending it requires setting the extension's package to org.apache.hadoop.mapred, which seems like poor form.
> I propose we make the two instrumentation classes public, so they can be extended wherever.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2043) TaskTrackerInstrumentation and
JobTrackerInstrumentation should be public
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philip Zeyliger updated MAPREDUCE-2043:
---------------------------------------
Attachment: MAPREDUCE-2043.patch.txt
Patch attached. Here are the changed lines, to save people some clicks:
{noformat}
-class JobTrackerInstrumentation {
+public class JobTrackerInstrumentation {
-class TaskTrackerInstrumentation {
+public class TaskTrackerInstrumentation {
{noformat}
> TaskTrackerInstrumentation and JobTrackerInstrumentation should be public
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-2043
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2043
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.22.0
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Attachments: MAPREDUCE-2043.patch.txt
>
>
> Hadoop administrators can specify classes to be loaded as "TaskTrackerInstrumentation" and "JobTrackerInstrumentation" implementations, which, roughly, define listeners on TT and JT events. Unfortunately, since the class has default access, extending it requires setting the extension's package to org.apache.hadoop.mapred, which seems like poor form.
> I propose we make the two instrumentation classes public, so they can be extended wherever.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2043) TaskTrackerInstrumentation and
JobTrackerInstrumentation should be public
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philip Zeyliger updated MAPREDUCE-2043:
---------------------------------------
Attachment: MAPREDUCE-2043.patch.txt
Chris and Luke,
Thanks for the context. I think the experimental developers ought to embed/extend Hadoop in a different package than org.apache.hadoop.mapred, so there's a reasonable argument for 'public', with the interface caveats. Agree wholeheartedly that this interface should be evolving. It's proven a convenient way, actually, to try some things out.
I've taken Luke's suggestion and added the interface annotations. I've "tested" this with "ant compile-core" only (ant compile breaks on mumak in contrib). Attached is that new patch.
-- Philip
> TaskTrackerInstrumentation and JobTrackerInstrumentation should be public
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-2043
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2043
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.22.0
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Attachments: MAPREDUCE-2043.patch.txt, MAPREDUCE-2043.patch.txt
>
>
> Hadoop administrators can specify classes to be loaded as "TaskTrackerInstrumentation" and "JobTrackerInstrumentation" implementations, which, roughly, define listeners on TT and JT events. Unfortunately, since the class has default access, extending it requires setting the extension's package to org.apache.hadoop.mapred, which seems like poor form.
> I propose we make the two instrumentation classes public, so they can be extended wherever.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.