[jira] [Created] (IGNITE-13996) JMX beans in a format not understood by Prometheus

["Stephen Darlington (Jira)" <ji...@apache.org>]

Stephen Darlington created IGNITE-13996:
-------------------------------------------

             Summary: JMX beans in a format not understood by Prometheus
                 Key: IGNITE-13996
                 URL: https://issues.apache.org/jira/browse/IGNITE-13996
             Project: Ignite
          Issue Type: Bug
    Affects Versions: 2.9.1, 2.8.1, 2.9
            Reporter: Stephen Darlington


When I connect Ignite to Prometheus' JMX exporter, I get the following exception:

{{ }}
{{ Jan 13, 2021 5:31:49 PM io.prometheus.jmx.shaded.io.prometheus.jmx.JmxCollector collect}}
{{ SEVERE: JMX scrape failed: java.lang.IllegalArgumentException: Not an Attribute: javax.management.openmbean.TabularDataSupport(tabularType=javax.management.openmbean.TabularType(name=org.apache.ignite.spi.systemview.view.ScanQueryView,rowType=javax.management.openmbean.CompositeType(name=org.apache.ignite.spi.systemview.view.ScanQueryView,items=((itemName=cacheGroupId,itemType=javax.management.openmbean.SimpleType(name=java.lang.Integer)),(itemName=cacheGroupName,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=cacheId,itemType=javax.management.openmbean.SimpleType(name=java.lang.Integer)),(itemName=cacheName,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=canceled,itemType=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),(itemName=duration,itemType=javax.management.openmbean.SimpleType(name=java.lang.Long)),(itemName=filter,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=keepBinary,itemType=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),(itemName=local,itemType=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),(itemName=originNodeId,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=pageSize,itemType=javax.management.openmbean.SimpleType(name=java.lang.Integer)),(itemName=partition,itemType=javax.management.openmbean.SimpleType(name=java.lang.Integer)),(itemName=queryId,itemType=javax.management.openmbean.SimpleType(name=java.lang.Long)),(itemName=startTime,itemType=javax.management.openmbean.SimpleType(name=java.lang.Long)),(itemName=subjectId,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=systemViewRowId,itemType=javax.management.openmbean.SimpleType(name=java.lang.Integer)),(itemName=taskName,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=topology,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=transformer,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)))),indexNames=(systemViewRowId)),contents={})}}
{{ at javax.management.AttributeList.adding(AttributeList.java:328)}}
{{ at javax.management.AttributeList.adding(AttributeList.java:335)}}
{{ at javax.management.AttributeList.asList(AttributeList.java:165)}}
{{ at io.prometheus.jmx.shaded.io.prometheus.jmx.JmxScraper.scrapeBean(JmxScraper.java:156)}}
{{ at io.prometheus.jmx.shaded.io.prometheus.jmx.JmxScraper.doScrape(JmxScraper.java:117)}}
{{ at io.prometheus.jmx.shaded.io.prometheus.jmx.JmxCollector.collect(JmxCollector.java:460)}}
{{ at io.prometheus.jmx.shaded.io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.findNextElement(CollectorRegistry.java:183)}}
{{ at io.prometheus.jmx.shaded.io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:216)}}
{{ at io.prometheus.jmx.shaded.io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:137)}}
{{ at io.prometheus.jmx.shaded.io.prometheus.client.exporter.common.TextFormat.write004(TextFormat.java:22)}}
{{ at io.prometheus.jmx.shaded.io.prometheus.client.exporter.HTTPServer$HTTPMetricHandler.handle(HTTPServer.java:59)}}
{{ at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)}}
{{ at sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:83)}}
{{ at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:82)}}
{{ at sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:675)}}
{{ at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)}}
{{ at sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:647)}}
{{ at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{ at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{ at java.lang.Thread.run(Thread.java:748)}}
{{  }}

And only the basic JVM metrics appear in Prometheus. A workaround would be to use OpenCensus, but a lot of people seem to prefer JMX.

It's not clear to me Ignite's JMX output is incorrect ([Prometheus' developers don't seem keen to resolve on their side|https://github.com/prometheus/jmx_exporter/issues/483]) but ideally, Ignite would work correctly with a common monitoring tool.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)