[jira] [Comment Edited] (FLINK-4532) Allow independent metrics reporter for tasks

["Renkai Ge (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-4532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15450946#comment-15450946 ] 

Renkai Ge edited comment on FLINK-4532 at 8/31/16 3:33 AM:
-----------------------------------------------------------

[~StephanEwen]
Yes, I think your proposal is actually a good way.To achieve this,we need
1. Add a command line option that users can give an additional flink-conf.yaml, the key-value pairs in the additional file may overwrite those in FLINK_HOME/conf/flink-conf.yaml.
2. Currently the command line client have two way to add user class,one is \-\-classpath option, but this option need us to put user jars on every machine of a YARN cluster or have a external http server before the JobManager started, not very friendly to use.One is the jar file for user task, but classes in this jar file can only be loaded by user task, not the JobManager or TaskManager. Should we make the usercodeJar loadable by JobManager and TaskManager or give an enhanced version of \-\-classpath option that automatically transfer jars in every YARN/Mesos/etc node and put them in classpath before JobManager/TaskManager started?


was (Author: renkaige):
[~StephanEwen]
Yes, I think your proposal is actually a good way.To achieve this,we need
1. Add a command line option that users can give an additional flink-conf.yaml, the key-value pairs in the additional file may overwrite those in FLINK_HOME/conf/flink-conf.yaml.
2. Currently the command line client have two way to add user class,one is '--classpath' option, but this option need us to put user jars on every machine of a YARN cluster or have a external http server before the JobManager started, not very friendly to use.One is the jar file for user task, but classes in this jar file can only be loaded by user task, not the JobManager or TaskManager. Should we make the usercodeJar loadable by JobManager and TaskManager or give an enhanced version of '--classpath' option that automatically transfer jars in every YARN/Mesos/etc node and put them in classpath before JobManager/TaskManager started?

> Allow independent metrics reporter for tasks
> --------------------------------------------
>
>                 Key: FLINK-4532
>                 URL: https://issues.apache.org/jira/browse/FLINK-4532
>             Project: Flink
>          Issue Type: Improvement
>          Components: Metrics
>            Reporter: Renkai Ge
>
> Currently,  JobManager, TaskManager and Task share the same configuration of reporters, and implementations of org.apache.flink.metrics.reporter.MetricReporter  should be in classpath before JobManager started. 
> In a cluster with multi tenancy, cluster administrator may want to monitor the cluster metrics in one place, and the owner of a application want to view the metrics of their application in somewhere else, and they may put custom MetricReporter in the application jar if the classpath don't provide one.
> I want to implement this by:
> 	1. Make it possible to load MetricReporter from userCodeClassLoader.
> 	2. Put new API into ExecutionConfig like {code}public long enableAdditionalMetricReporters(List<String> classNames,boolean disableGlobalMetricReporters);{code}
> I will begin to work on this if you guys agrees.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)