[jira] [Updated] (AMBARI-4881) Clean up JMXPropertyProvider hacks for STORM metrics

["Dmitry Lysnichenko (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMBARI-4881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitry Lysnichenko updated AMBARI-4881:
---------------------------------------

    Description: 
h2. The task
Clean up a JMXPropertyProvider hacks for STORM metrics
Also,
- probably, create a separate provider for rest api 
- current implementation is only addressing /api/cluster/summary from Storm.  Allow accessing other urls like  /api/supervisors/summary etc
- When we refactor (assuming using the HttpProxyPropertyProvider), make that be able to take any type of reader.  Also, we want to move to Gson, not Jackson.
- Is it possible to set the port of the STORM_REST_API component using the UI?  If so, then this will result in a failed query.

h3. Create RestPropertyProvider
RestPropertyProvider implementation will be registered at StackDefinedPropertyProvider via reflection ( org.apache.ambari.server.controller.internal.StackDefinedPropertyProvider#getDelegate )

h3. Add new metrics type "rest" 
{code}
{
  "type": "rest",
  "properties": {
     "default_port": "8745" # Storm port is always hardcoded, so we will just use the default value
     "port_config_type": "storm-site" # Just example, not needed for Storm
     "port_property_name": "storm.port" # Just example, not needed for Storm
  }
  "metrics": {
    "metrics/api/cluster/summary/nimbus.uptime": {
      "metric": "api/cluster/summary/nimbus.uptimenimbus.uptime",
      "pointInTime": true,
      "temporal": true
    },
   ...
{code}
Metric group properties define port-related settings. If "port_property" and "relevant_config" keys are defined, provider will try to extract port setting from the service config. Otherwise, provider will use the default port. RestPropertyProvider maintains a cache of per-component port numbers just like we do at org.apache.ambari.server.controller.internal.AbstractProviderModule#getPort
Metric id "metrics/api/cluster/summary/nimbus.uptime" contains both url and property name. We can not use sole url as metric id because defining few metrics with the same id is not possible.

h3. Adding to RestPropertyProvider support of nested json properties.
Given a json file like:
{code}
{
  "a" : "b",
  "c" : {
    "d" : "e",
  }
}
{code}
We may get "e" value by defining metric in form metrics/api/cluster/summary/c#d where "api/cluster/summary" is a json url, and "c#d" points to property "d" nested under property "c". Any nesting depth is supported. Accessing properties inside json value lists is not supported (I don't see any use case for that).
Going this way, we will be able to access metrics defined at (almost) arbitrary json files at arbitrary ports/urls. 

  was:
h2. The task
Clean up a JMXPropertyProvider hacks for STORM metrics
Also,
- probably, create a separate provider for rest api 
- current implementation is only addressing /api/cluster/summary from Storm.  Allow accessing other urls like  /api/supervisors/summary etc
- When we refactor (assuming using the HttpProxyPropertyProvider), make that be able to take any type of reader.  Also, we want to move to Gson, not Jackson.
- Is it possible to set the port of the STORM_REST_API component using the UI?  If so, then this will result in a failed query.

h3. Create RestPropertyProvider
RestPropertyProvider implementation will be registered at StackDefinedPropertyProvider via reflection ( org.apache.ambari.server.controller.internal.StackDefinedPropertyProvider#getDelegate )

h3. Add new metrics type "rest" 
{code}
{
  "type": "rest",
  "properties": {
     "default_port": "8745" # Storm port is always hardcoded, so we will just use the default value
     "port_config_type": "storm-site" # Just example, not needed for Storm
     "port_property_name": "storm.port" # Just example, not needed for Storm
  }
  "metrics": {
    "metrics/api/cluster/summary/nimbus.uptime": {
      "metric": "nimbus.uptime",
      "pointInTime": true,
      "temporal": true
    },
   ...
{code}
Metric group properties define port-related settings. If "port_property" and "relevant_config" keys are defined, provider will try to extract port setting from the service config. Otherwise, provider will use the default port. RestPropertyProvider maintains a cache of per-component port numbers just like we do at org.apache.ambari.server.controller.internal.AbstractProviderModule#getPort
Metric id "metrics/api/cluster/summary/nimbus.uptime" contains both url and property name. We can not use sole url as metric id because defining few metrics with the same id is not possible.

h3. Adding to RestPropertyProvider support of nested json properties.
Given a json file like:
{code}
{
  "a" : "b",
  "c" : {
    "d" : "e",
  }
}
{code}
We may get "e" value by defining metric in form metrics/api/cluster/summary/c#d where "api/cluster/summary" is a json url, and "c#d" points to property "d" nested under property "c". Any nesting depth is supported. Accessing properties inside json value lists is not supported (I don't see any use case for that).
Going this way, we will be able to access metrics defined at (almost) arbitrary json files at arbitrary ports/urls. 


> Clean up JMXPropertyProvider hacks for STORM metrics
> ----------------------------------------------------
>
>                 Key: AMBARI-4881
>                 URL: https://issues.apache.org/jira/browse/AMBARI-4881
>             Project: Ambari
>          Issue Type: Task
>          Components: controller, test
>    Affects Versions: 1.5.0
>            Reporter: Dmitry Lysnichenko
>            Assignee: Dmitry Lysnichenko
>             Fix For: 1.6.0
>
>
> h2. The task
> Clean up a JMXPropertyProvider hacks for STORM metrics
> Also,
> - probably, create a separate provider for rest api 
> - current implementation is only addressing /api/cluster/summary from Storm.  Allow accessing other urls like  /api/supervisors/summary etc
> - When we refactor (assuming using the HttpProxyPropertyProvider), make that be able to take any type of reader.  Also, we want to move to Gson, not Jackson.
> - Is it possible to set the port of the STORM_REST_API component using the UI?  If so, then this will result in a failed query.
> h3. Create RestPropertyProvider
> RestPropertyProvider implementation will be registered at StackDefinedPropertyProvider via reflection ( org.apache.ambari.server.controller.internal.StackDefinedPropertyProvider#getDelegate )
> h3. Add new metrics type "rest" 
> {code}
> {
>   "type": "rest",
>   "properties": {
>      "default_port": "8745" # Storm port is always hardcoded, so we will just use the default value
>      "port_config_type": "storm-site" # Just example, not needed for Storm
>      "port_property_name": "storm.port" # Just example, not needed for Storm
>   }
>   "metrics": {
>     "metrics/api/cluster/summary/nimbus.uptime": {
>       "metric": "api/cluster/summary/nimbus.uptimenimbus.uptime",
>       "pointInTime": true,
>       "temporal": true
>     },
>    ...
> {code}
> Metric group properties define port-related settings. If "port_property" and "relevant_config" keys are defined, provider will try to extract port setting from the service config. Otherwise, provider will use the default port. RestPropertyProvider maintains a cache of per-component port numbers just like we do at org.apache.ambari.server.controller.internal.AbstractProviderModule#getPort
> Metric id "metrics/api/cluster/summary/nimbus.uptime" contains both url and property name. We can not use sole url as metric id because defining few metrics with the same id is not possible.
> h3. Adding to RestPropertyProvider support of nested json properties.
> Given a json file like:
> {code}
> {
>   "a" : "b",
>   "c" : {
>     "d" : "e",
>   }
> }
> {code}
> We may get "e" value by defining metric in form metrics/api/cluster/summary/c#d where "api/cluster/summary" is a json url, and "c#d" points to property "d" nested under property "c". Any nesting depth is supported. Accessing properties inside json value lists is not supported (I don't see any use case for that).
> Going this way, we will be able to access metrics defined at (almost) arbitrary json files at arbitrary ports/urls. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)