You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by "Nordstrom, Karl" <kx...@psu.edu> on 2022/03/15 15:05:41 UTC
ActiveMQ Telegraf errors
Hello,
After upgrading from ActiveMQ 5.16.0 to 5.16.4, we are getting telegraf errors.
amq-p01 ~]$ sudo systemctl status telegraf | cat
● telegraf.service - The plugin-driven server agent for reporting metrics into InfluxDB
Loaded: loaded (/usr/lib/systemd/system/telegraf.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2022-03-15 06:02:36 EDT; 4h 42min ago
Docs: https://github.com/influxdata/telegraf
Main PID: 1219 (telegraf)
CGroup: /system.slice/telegraf.service
└─1219 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d
Mar 15 10:43:10 amq-p02.blue.psu.edu telegraf[1219]: 2022-03-15T14:43:10Z E! [inputs.jolokia2_agent]: Error in plugin: Unable to gather metrics for http://localhost:8161/api/jolokia/read: Post http://localhost:8161/api/jolokia/read/read: dial tcp [::1]:8161: connect: connection refused
I'm seeing errors like this in /var/log/messages too:
2022-03-15T09:55:40.372633-04:00 amq-p01.blue.psu.edu telegraf: 2022-03-15T13:55:40Z E! [inputs.jolokia2_agent]: Error in plugin: Unable to gather metrics for http://localhost:8161/api/jolokia/read: Response from url "http://localhost:8161/api/jolokia/read" has status code 401 (Unauthorized), expected 200 (OK)
Here is our telegraf conf for activemq
amq-p01 ~]$ sudo cat /etc/telegraf/telegraf.d/activemq-prod-stats.conf
[[inputs.jolokia2_agent]]
urls = ["http://localhost:8161/api/jolokia/read"]
[[inputs.jolokia2_agent.metric]]
name = "ActiveMQ"
mbean = "org.apache.activemq:brokerName=amq-prod-cluster,destinationName=*,destinationType=Queue,type=Broker"
tag_keys = ["destinationName"]
[[inputs.jolokia2_agent.metric]]
name = "jvm_memory"
mbean = "java.lang:type=Memory"
paths = ["HeapMemoryUsage", "NonHeapMemoryUsage", "ObjectPendingFinalizationCount"]
[[inputs.jolokia2_agent.metric]]
name = "jvm_os"
mbean = "java.lang:type=OperatingSystem"
paths = ["MaxFileDescriptorCount", "OpenFileDescriptorCount"]
[[inputs.jolokia2_agent.metric]]
name = "jvm_threading"
mbean = "java.lang:type=Threading"
paths = ["DaemonThreadCount", "PeakThreadCount", "CurrentThreadUserTime", "TotalStartedThreadCount", "CurrentThreadCpuTime", "ThreadCount"]
[[inputs.jolokia2_agent.metric]]
name = "jvm_class_loading"
mbean = "java.lang:type=ClassLoading"
paths = ["TotalLoadedClassCount", "UnloadedClassCount", "LoadedClassCount"]
[[inputs.jolokia2_agent.metric]]
name = "jvm_garbage_collector"
mbean = "java.lang:name=*,type=GarbageCollector"
tag_keys = ["name"]
Are the permissions for this file correct?
amq-p01 ~]$ sudo ls -l /etc/telegraf/telegraf.d/activemq-prod-stats.conf
-r--r----- 1 root telegraf 1173 Jan 26 2021 /etc/telegraf/telegraf.d/activemq-prod-stats.conf
---
Karl Nordström
Systems Administrator
Penn State IT | Application Platforms
Re: ActiveMQ Telegraf errors
Posted by "Nordstrom, Karl" <kx...@psu.edu>.
JB,
Thanks for the email response. I searched the web for how to disable authentication on jolokia. I'm finding lots of documentation that isn't very helpful.
The only thing I knew about jolokia until recently was this message in the activemq.log
2022-03-15 06:17:34,355 | INFO | ActiveMQ Jolokia REST API available at http://0.0.0.0:8161/api/jolokia/ | org.apache.activemq.web.WebConsoleStarter | main
Then I can log into the admin console with the admin user id and password.
I found this, but I don't know where the jolokia-access.xml is supposed to go.
https://github.com/rhuss/jolokia/blob/master/src/site/resources/templates/jolokia-access.xml
This page has a config file, but it doesn't look like our /etc/telegraf/telegraf.d/activemq-prod-stats.conf
https://www.influxdata.com/integration/activemq/#:~:text=Regardless%20of%20how%20you%20choose%20to%20use%20the,and%20subscriber%20metrics%20using%20the%20ActiveMQ%20Console%20API
Could you please give me more specifics on how to setup credentials for jolokia?
Thanks,
---
Karl Nordström
Systems Administrator
Penn State IT | Application Platforms
________________________________
From: Jean-Baptiste Onofré <jb...@nanthrax.net>
Sent: Tuesday, March 15, 2022 11:58 AM
To: users@activemq.apache.org <us...@activemq.apache.org>
Subject: Re: ActiveMQ Telegraf errors
Hi,
It's maybe related to security enforcement we added in 5.16.4. Before,
jolokia access was not "secure" (open access). In 5.16.4, credential is
required to access jolokia, that's maybe related. Did you try to disable
authentication on jolokia ?
Regards
JB
On Tue, Mar 15, 2022 at 4:05 PM Nordstrom, Karl <kx...@psu.edu> wrote:
> Hello,
>
> After upgrading from ActiveMQ 5.16.0 to 5.16.4, we are getting telegraf
> errors.
>
> amq-p01 ~]$ sudo systemctl status telegraf | cat
> ● telegraf.service - The plugin-driven server agent for reporting metrics
> into InfluxDB
> Loaded: loaded (/usr/lib/systemd/system/telegraf.service; enabled;
> vendor preset: disabled)
> Active: active (running) since Tue 2022-03-15 06:02:36 EDT; 4h 42min ago
> Docs: https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Finfluxdata%2Ftelegraf&data=04%7C01%7Ckxn2%40psu.edu%7Cf364c42f71df415031d808da069cbcd2%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C637829568027859671%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=feA14UO%2B1ZA8VnlXka3Zd8oGRuqYHoLb8xIW8yFuWW0%3D&reserved=0
> Main PID: 1219 (telegraf)
> CGroup: /system.slice/telegraf.service
> └─1219 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf
> -config-directory /etc/telegraf/telegraf.d
>
> Mar 15 10:43:10 amq-p02.blue.psu.edu telegraf[1219]: 2022-03-15T14:43:10Z
> E! [inputs.jolokia2_agent]: Error in plugin: Unable to gather metrics for
> http://localhost:8161/api/jolokia/read: Post
> http://localhost:8161/api/jolokia/read/read: dial tcp [::1]:8161:
> connect: connection refused
>
> I'm seeing errors like this in /var/log/messages too:
>
> 2022-03-15T09:55:40.372633-04:00 amq-p01.blue.psu.edu telegraf:
> 2022-03-15T13:55:40Z E! [inputs.jolokia2_agent]: Error in plugin: Unable to
> gather metrics for http://localhost:8161/api/jolokia/read: Response from
> url "http://localhost:8161/api/jolokia/read" has status code 401
> (Unauthorized), expected 200 (OK)
>
> Here is our telegraf conf for activemq
>
> amq-p01 ~]$ sudo cat /etc/telegraf/telegraf.d/activemq-prod-stats.conf
> [[inputs.jolokia2_agent]]
> urls = ["http://localhost:8161/api/jolokia/read"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "ActiveMQ"
> mbean =
> "org.apache.activemq:brokerName=amq-prod-cluster,destinationName=*,destinationType=Queue,type=Broker"
> tag_keys = ["destinationName"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "jvm_memory"
> mbean = "java.lang:type=Memory"
> paths = ["HeapMemoryUsage", "NonHeapMemoryUsage",
> "ObjectPendingFinalizationCount"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "jvm_os"
> mbean = "java.lang:type=OperatingSystem"
> paths = ["MaxFileDescriptorCount", "OpenFileDescriptorCount"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "jvm_threading"
> mbean = "java.lang:type=Threading"
> paths = ["DaemonThreadCount", "PeakThreadCount",
> "CurrentThreadUserTime", "TotalStartedThreadCount", "CurrentThreadCpuTime",
> "ThreadCount"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "jvm_class_loading"
> mbean = "java.lang:type=ClassLoading"
> paths = ["TotalLoadedClassCount", "UnloadedClassCount",
> "LoadedClassCount"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "jvm_garbage_collector"
> mbean = "java.lang:name=*,type=GarbageCollector"
> tag_keys = ["name"]
>
> Are the permissions for this file correct?
>
> amq-p01 ~]$ sudo ls -l /etc/telegraf/telegraf.d/activemq-prod-stats.conf
> -r--r----- 1 root telegraf 1173 Jan 26 2021
> /etc/telegraf/telegraf.d/activemq-prod-stats.conf
>
>
>
> ---
>
> Karl Nordström
>
> Systems Administrator
>
> Penn State IT | Application Platforms
>
Re: ActiveMQ Telegraf errors
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi,
It's maybe related to security enforcement we added in 5.16.4. Before,
jolokia access was not "secure" (open access). In 5.16.4, credential is
required to access jolokia, that's maybe related. Did you try to disable
authentication on jolokia ?
Regards
JB
On Tue, Mar 15, 2022 at 4:05 PM Nordstrom, Karl <kx...@psu.edu> wrote:
> Hello,
>
> After upgrading from ActiveMQ 5.16.0 to 5.16.4, we are getting telegraf
> errors.
>
> amq-p01 ~]$ sudo systemctl status telegraf | cat
> ● telegraf.service - The plugin-driven server agent for reporting metrics
> into InfluxDB
> Loaded: loaded (/usr/lib/systemd/system/telegraf.service; enabled;
> vendor preset: disabled)
> Active: active (running) since Tue 2022-03-15 06:02:36 EDT; 4h 42min ago
> Docs: https://github.com/influxdata/telegraf
> Main PID: 1219 (telegraf)
> CGroup: /system.slice/telegraf.service
> └─1219 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf
> -config-directory /etc/telegraf/telegraf.d
>
> Mar 15 10:43:10 amq-p02.blue.psu.edu telegraf[1219]: 2022-03-15T14:43:10Z
> E! [inputs.jolokia2_agent]: Error in plugin: Unable to gather metrics for
> http://localhost:8161/api/jolokia/read: Post
> http://localhost:8161/api/jolokia/read/read: dial tcp [::1]:8161:
> connect: connection refused
>
> I'm seeing errors like this in /var/log/messages too:
>
> 2022-03-15T09:55:40.372633-04:00 amq-p01.blue.psu.edu telegraf:
> 2022-03-15T13:55:40Z E! [inputs.jolokia2_agent]: Error in plugin: Unable to
> gather metrics for http://localhost:8161/api/jolokia/read: Response from
> url "http://localhost:8161/api/jolokia/read" has status code 401
> (Unauthorized), expected 200 (OK)
>
> Here is our telegraf conf for activemq
>
> amq-p01 ~]$ sudo cat /etc/telegraf/telegraf.d/activemq-prod-stats.conf
> [[inputs.jolokia2_agent]]
> urls = ["http://localhost:8161/api/jolokia/read"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "ActiveMQ"
> mbean =
> "org.apache.activemq:brokerName=amq-prod-cluster,destinationName=*,destinationType=Queue,type=Broker"
> tag_keys = ["destinationName"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "jvm_memory"
> mbean = "java.lang:type=Memory"
> paths = ["HeapMemoryUsage", "NonHeapMemoryUsage",
> "ObjectPendingFinalizationCount"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "jvm_os"
> mbean = "java.lang:type=OperatingSystem"
> paths = ["MaxFileDescriptorCount", "OpenFileDescriptorCount"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "jvm_threading"
> mbean = "java.lang:type=Threading"
> paths = ["DaemonThreadCount", "PeakThreadCount",
> "CurrentThreadUserTime", "TotalStartedThreadCount", "CurrentThreadCpuTime",
> "ThreadCount"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "jvm_class_loading"
> mbean = "java.lang:type=ClassLoading"
> paths = ["TotalLoadedClassCount", "UnloadedClassCount",
> "LoadedClassCount"]
>
> [[inputs.jolokia2_agent.metric]]
> name = "jvm_garbage_collector"
> mbean = "java.lang:name=*,type=GarbageCollector"
> tag_keys = ["name"]
>
> Are the permissions for this file correct?
>
> amq-p01 ~]$ sudo ls -l /etc/telegraf/telegraf.d/activemq-prod-stats.conf
> -r--r----- 1 root telegraf 1173 Jan 26 2021
> /etc/telegraf/telegraf.d/activemq-prod-stats.conf
>
>
>
> ---
>
> Karl Nordström
>
> Systems Administrator
>
> Penn State IT | Application Platforms
>