You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Vilius Šumskas <vi...@rivile.lt> on 2023/01/11 11:55:09 UTC
reloading TLS certificate for Jolokia
Hi,
since because of https://issues.apache.org/jira/browse/ARTEMIS-3117 ActiveMQ Artemis doesn‘t reload TLS certificates automatically anymore I‘m wondering, what is the correct way to reload certificates for embedded Jolokia Console?
I have reloaded keystore for acceptors via console using reload() on „artemis-ssl“ acceptor, but I just cannot figure out a way to reload it for console itself.
--
Best Regards,
Vilius Šumskas
Rivile
IT manager
Re: reloading TLS certificate for Jolokia
Posted by Justin Bertram <jb...@apache.org>.
Glad to help.
It's not surprising that this would cause trouble. It's certainly not
something I would ever try in a production situation. Even the individual
who commented on the Oracle forum [1] stated:
Short answer, restart app server after upgrading JDK to be safe. If you
want to be on the edge and live dangerously, try it without restarting.
Production is not really where you want to be "on the edge and live
dangerously."
Justin
[1]
https://community.oracle.com/tech/developers/discussion/4084875/upgrading-jvm-while-application-server-running
On Thu, Jan 12, 2023 at 1:31 PM Vilius Šumskas <vi...@rivile.lt>
wrote:
> Just leaving this for others.
>
> My theory is confirmed, if you have upgraded OpenJDK under running live
> Artemis broker and didn't restart it afterwards your
> restartEmbeddedWebServer() command will fail. Hawtio console will be
> stopped but won't automatically re-start.
>
> Reproducible steps under Rocky Linux 9:
> 1. Login into running Hawtio console, click on Operations tab and Exexute
> restartEmbeddedWebServer(). The user gets logged out but after few seconds
> console is available again, which is correct behaviour.
> 2. Now login into SSH on the server running Artemis broker and run "dnf
> update java-17-openjdk-headless" or "dnf downgrade
> java-17-openjdk-headless".
> 3. Login back into running Hawtio console, click on Operations tab and
> Exexute restartEmbeddedWebServer() again. The console is now down and never
> comes up.
>
> The moral of the story - never upgrade JDK if you are unable to restart
> Artemis broker at the same time. Else you will have problems later, for
> example when reloading Hawtio console for TLS certificate change.
>
> Thank you for your help Justin!
>
> --
> Best Regards,
> Vilius
>
> -----Original Message-----
> From: Vilius Šumskas <vi...@rivile.lt>
> Sent: Thursday, January 12, 2023 12:48 PM
> To: users@activemq.apache.org
> Subject: RE: reloading TLS certificate for Jolokia
>
> Yes, the upgrade is removing older Java version, however this is only for
> minor versions. In theory internets say that it should depend on software
> if that's safe or not
> https://community.oracle.com/tech/developers/discussion/4084875/upgrading-jvm-while-application-server-running
> For example in Jenkins case it's not safe
> https://stackoverflow.com/questions/35758284/does-tomcat-need-to-be-restarted-after-a-jdk-is-updated
> For Artemis I did this multiple time and it always seemed fine.
>
> I see that now it's more complicated to bring everything up, so I guess I
> will make a maintenance window on production and restart the broker
> completely.
>
> I will test my theory regarding JDK upgrade and console restart a little
> bit later and report the findings.
>
> --
> Vilius
>
> -----Original Message-----
> From: Justin Bertram <jb...@apache.org>
> Sent: Wednesday, January 11, 2023 11:20 PM
> To: users@activemq.apache.org
> Subject: Re: reloading TLS certificate for Jolokia
>
> > Could it be that our Artemis instance is still using older JDK which
> exist only in memory, and console "inherits" that by trying to start
> itself, and then fails?
>
> I don't have any experience with such a use-case so it's hard to say. The
> "console" isn't fundamentally different from any other part of the broker.
> It's all just Java classes. Therefore, I wouldn't expect such a failure to
> be isolated there. If the rest of the broker is working fine I would expect
> the embedded web server to work fine as well.
>
> Are you _removing_ the version of Java that the broker is using during the
> upgrade or are you just installing a new version alongside the existing
> version? If the former, that seems dangerous in my opinion.
>
> > Is Artemis supposed to reload management.xml automatically...
>
> No. The contents of management.xml are not reloaded automatically.
>
> > ...do we have a chicken-egg problem now; cannot enable JMX because JMX
> > is
> down?
>
> You do have a chicken-egg problem with using JMX remotely. However, you
> could still theoretically use it locally through a direct JVM connection.
>
> You also still have the option of using management messages [1]. You can
> find an example of their use in the examples/features/standard/management
> directory shipped with the broker.
>
>
> Justin
>
> [1]
>
> https://activemq.apache.org/components/artemis/documentation/latest/management.html#using-management-message-api
>
>
>
> On Wed, Jan 11, 2023 at 2:49 PM Vilius Šumskas <vi...@rivile.lt>
> wrote:
>
> > No, I don't see any of these messages in the log. I have posted full
> > log what I see.
> >
> > This is production system so getting threadump is not so easy, but I
> > have decided to test this on our testing environment which runs newest
> 2.27.1.
> > Running restartEmbeddedWebServer() via Hawtio console on that
> > environment worked just fine.
> >
> > I have remembered one thing though. We have upgraded OpenJDK on our
> > production environment some time ago, but never restarted Artemis.
> > Usually this doesn't produce any issues, but if we decide to restart
> > Artemis after JDK upgrade it usually produces some errors about java
> > binaries cannot be found. As if Artemis was still loaded on older JDK
> > version and stopping it unloads these non-existing older JDK binaries
> from memory.
> > Could it be that our Artemis instance is still using older JDK which
> > exist only in memory, and console "inherits" that by trying to start
> > itself, and then fails?
> >
> > Anyway, I see that JMX Remote is disabled by default. I have enabled
> > JMX connector in management.xml, but I don't see port 1099 coming up.
> > Is Artemis supposed to reload management.xml automatically, or do we
> > have a chicken-egg problem now; cannot enable JMX because JMX is down?
> >
> > --
> > Vilius
> >
> > -----Original Message-----
> > From: Justin Bertram <jb...@apache.org>
> > Sent: Wednesday, January 11, 2023 6:46 PM
> > To: users@activemq.apache.org
> > Subject: Re: reloading TLS certificate for Jolokia
> >
> > Those exceptions look internal to Jetty. I wouldn't expect them to
> > necessarily be a problem for the restart process. Perhaps it was
> > handling a request at the moment you tried to restart it.
> >
> > Do you ever see any of these messages in the log:
> >
> > Stopped embedded web server
> >
> > or
> >
> > Embedded web server not restarted in x milliseconds
> >
> > or
> >
> > Embedded web server restart failed
> >
> > If not, could you perhaps get a thread dump?
> >
> > In any event, if you want to start the embedded web server you can do
> > so using JMX directly (e.g. via JConsole). Just invoke the
> > startEmbeddedWebServer operation on the ActiveMQServerControl MBean.
> >
> >
> > Justin
> >
> > On Wed, Jan 11, 2023 at 6:41 AM Vilius Šumskas
> > <vi...@rivile.lt>
> > wrote:
> >
> > > Well, I have found " restartEmbeddedWebServer()" and tried to use
> > > that from the console, however now the console went down and doesn't
> come up.
> > > The logs say: https://p.defau.lt/?C6ssFYW0JSJiZX1yszP0yQ
> > >
> > > Jolokia API is also down now. Is there a way to start embedded web
> > > server without restarting the broker?
> > >
> > > --
> > > Vilius
> > >
> > > -----Original Message-----
> > > From: Vilius Šumskas <vi...@rivile.lt>
> > > Sent: Wednesday, January 11, 2023 1:55 PM
> > > To: users@activemq.apache.org
> > > Subject: reloading TLS certificate for Jolokia
> > >
> > > Hi,
> > >
> > > since because of https://issues.apache.org/jira/browse/ARTEMIS-3117
> > > ActiveMQ Artemis doesn‘t reload TLS certificates automatically
> > > anymore I‘m wondering, what is the correct way to reload
> > > certificates for embedded Jolokia Console?
> > >
> > > I have reloaded keystore for acceptors via console using reload() on
> > > „artemis-ssl“ acceptor, but I just cannot figure out a way to reload
> > > it for console itself.
> > >
> > > --
> > > Best Regards,
> > >
> > > Vilius Šumskas
> > > Rivile
> > > IT manager
> > >
> > >
> >
>
RE: reloading TLS certificate for Jolokia
Posted by Vilius Šumskas <vi...@rivile.lt>.
Just leaving this for others.
My theory is confirmed, if you have upgraded OpenJDK under running live Artemis broker and didn't restart it afterwards your restartEmbeddedWebServer() command will fail. Hawtio console will be stopped but won't automatically re-start.
Reproducible steps under Rocky Linux 9:
1. Login into running Hawtio console, click on Operations tab and Exexute restartEmbeddedWebServer(). The user gets logged out but after few seconds console is available again, which is correct behaviour.
2. Now login into SSH on the server running Artemis broker and run "dnf update java-17-openjdk-headless" or "dnf downgrade java-17-openjdk-headless".
3. Login back into running Hawtio console, click on Operations tab and Exexute restartEmbeddedWebServer() again. The console is now down and never comes up.
The moral of the story - never upgrade JDK if you are unable to restart Artemis broker at the same time. Else you will have problems later, for example when reloading Hawtio console for TLS certificate change.
Thank you for your help Justin!
--
Best Regards,
Vilius
-----Original Message-----
From: Vilius Šumskas <vi...@rivile.lt>
Sent: Thursday, January 12, 2023 12:48 PM
To: users@activemq.apache.org
Subject: RE: reloading TLS certificate for Jolokia
Yes, the upgrade is removing older Java version, however this is only for minor versions. In theory internets say that it should depend on software if that's safe or not https://community.oracle.com/tech/developers/discussion/4084875/upgrading-jvm-while-application-server-running
For example in Jenkins case it's not safe https://stackoverflow.com/questions/35758284/does-tomcat-need-to-be-restarted-after-a-jdk-is-updated
For Artemis I did this multiple time and it always seemed fine.
I see that now it's more complicated to bring everything up, so I guess I will make a maintenance window on production and restart the broker completely.
I will test my theory regarding JDK upgrade and console restart a little bit later and report the findings.
--
Vilius
-----Original Message-----
From: Justin Bertram <jb...@apache.org>
Sent: Wednesday, January 11, 2023 11:20 PM
To: users@activemq.apache.org
Subject: Re: reloading TLS certificate for Jolokia
> Could it be that our Artemis instance is still using older JDK which
exist only in memory, and console "inherits" that by trying to start itself, and then fails?
I don't have any experience with such a use-case so it's hard to say. The "console" isn't fundamentally different from any other part of the broker.
It's all just Java classes. Therefore, I wouldn't expect such a failure to be isolated there. If the rest of the broker is working fine I would expect the embedded web server to work fine as well.
Are you _removing_ the version of Java that the broker is using during the upgrade or are you just installing a new version alongside the existing version? If the former, that seems dangerous in my opinion.
> Is Artemis supposed to reload management.xml automatically...
No. The contents of management.xml are not reloaded automatically.
> ...do we have a chicken-egg problem now; cannot enable JMX because JMX
> is
down?
You do have a chicken-egg problem with using JMX remotely. However, you could still theoretically use it locally through a direct JVM connection.
You also still have the option of using management messages [1]. You can find an example of their use in the examples/features/standard/management
directory shipped with the broker.
Justin
[1]
https://activemq.apache.org/components/artemis/documentation/latest/management.html#using-management-message-api
On Wed, Jan 11, 2023 at 2:49 PM Vilius Šumskas <vi...@rivile.lt>
wrote:
> No, I don't see any of these messages in the log. I have posted full
> log what I see.
>
> This is production system so getting threadump is not so easy, but I
> have decided to test this on our testing environment which runs newest 2.27.1.
> Running restartEmbeddedWebServer() via Hawtio console on that
> environment worked just fine.
>
> I have remembered one thing though. We have upgraded OpenJDK on our
> production environment some time ago, but never restarted Artemis.
> Usually this doesn't produce any issues, but if we decide to restart
> Artemis after JDK upgrade it usually produces some errors about java
> binaries cannot be found. As if Artemis was still loaded on older JDK
> version and stopping it unloads these non-existing older JDK binaries from memory.
> Could it be that our Artemis instance is still using older JDK which
> exist only in memory, and console "inherits" that by trying to start
> itself, and then fails?
>
> Anyway, I see that JMX Remote is disabled by default. I have enabled
> JMX connector in management.xml, but I don't see port 1099 coming up.
> Is Artemis supposed to reload management.xml automatically, or do we
> have a chicken-egg problem now; cannot enable JMX because JMX is down?
>
> --
> Vilius
>
> -----Original Message-----
> From: Justin Bertram <jb...@apache.org>
> Sent: Wednesday, January 11, 2023 6:46 PM
> To: users@activemq.apache.org
> Subject: Re: reloading TLS certificate for Jolokia
>
> Those exceptions look internal to Jetty. I wouldn't expect them to
> necessarily be a problem for the restart process. Perhaps it was
> handling a request at the moment you tried to restart it.
>
> Do you ever see any of these messages in the log:
>
> Stopped embedded web server
>
> or
>
> Embedded web server not restarted in x milliseconds
>
> or
>
> Embedded web server restart failed
>
> If not, could you perhaps get a thread dump?
>
> In any event, if you want to start the embedded web server you can do
> so using JMX directly (e.g. via JConsole). Just invoke the
> startEmbeddedWebServer operation on the ActiveMQServerControl MBean.
>
>
> Justin
>
> On Wed, Jan 11, 2023 at 6:41 AM Vilius Šumskas
> <vi...@rivile.lt>
> wrote:
>
> > Well, I have found " restartEmbeddedWebServer()" and tried to use
> > that from the console, however now the console went down and doesn't come up.
> > The logs say: https://p.defau.lt/?C6ssFYW0JSJiZX1yszP0yQ
> >
> > Jolokia API is also down now. Is there a way to start embedded web
> > server without restarting the broker?
> >
> > --
> > Vilius
> >
> > -----Original Message-----
> > From: Vilius Šumskas <vi...@rivile.lt>
> > Sent: Wednesday, January 11, 2023 1:55 PM
> > To: users@activemq.apache.org
> > Subject: reloading TLS certificate for Jolokia
> >
> > Hi,
> >
> > since because of https://issues.apache.org/jira/browse/ARTEMIS-3117
> > ActiveMQ Artemis doesn‘t reload TLS certificates automatically
> > anymore I‘m wondering, what is the correct way to reload
> > certificates for embedded Jolokia Console?
> >
> > I have reloaded keystore for acceptors via console using reload() on
> > „artemis-ssl“ acceptor, but I just cannot figure out a way to reload
> > it for console itself.
> >
> > --
> > Best Regards,
> >
> > Vilius Šumskas
> > Rivile
> > IT manager
> >
> >
>
RE: reloading TLS certificate for Jolokia
Posted by Vilius Šumskas <vi...@rivile.lt>.
Yes, the upgrade is removing older Java version, however this is only for minor versions. In theory internets say that it should depend on software if that's safe or not https://community.oracle.com/tech/developers/discussion/4084875/upgrading-jvm-while-application-server-running
For example in Jenkins case it's not safe https://stackoverflow.com/questions/35758284/does-tomcat-need-to-be-restarted-after-a-jdk-is-updated
For Artemis I did this multiple time and it always seemed fine.
I see that now it's more complicated to bring everything up, so I guess I will make a maintenance window on production and restart the broker completely.
I will test my theory regarding JDK upgrade and console restart a little bit later and report the findings.
--
Vilius
-----Original Message-----
From: Justin Bertram <jb...@apache.org>
Sent: Wednesday, January 11, 2023 11:20 PM
To: users@activemq.apache.org
Subject: Re: reloading TLS certificate for Jolokia
> Could it be that our Artemis instance is still using older JDK which
exist only in memory, and console "inherits" that by trying to start itself, and then fails?
I don't have any experience with such a use-case so it's hard to say. The "console" isn't fundamentally different from any other part of the broker.
It's all just Java classes. Therefore, I wouldn't expect such a failure to be isolated there. If the rest of the broker is working fine I would expect the embedded web server to work fine as well.
Are you _removing_ the version of Java that the broker is using during the upgrade or are you just installing a new version alongside the existing version? If the former, that seems dangerous in my opinion.
> Is Artemis supposed to reload management.xml automatically...
No. The contents of management.xml are not reloaded automatically.
> ...do we have a chicken-egg problem now; cannot enable JMX because JMX
> is
down?
You do have a chicken-egg problem with using JMX remotely. However, you could still theoretically use it locally through a direct JVM connection.
You also still have the option of using management messages [1]. You can find an example of their use in the examples/features/standard/management
directory shipped with the broker.
Justin
[1]
https://activemq.apache.org/components/artemis/documentation/latest/management.html#using-management-message-api
On Wed, Jan 11, 2023 at 2:49 PM Vilius Šumskas <vi...@rivile.lt>
wrote:
> No, I don't see any of these messages in the log. I have posted full
> log what I see.
>
> This is production system so getting threadump is not so easy, but I
> have decided to test this on our testing environment which runs newest 2.27.1.
> Running restartEmbeddedWebServer() via Hawtio console on that
> environment worked just fine.
>
> I have remembered one thing though. We have upgraded OpenJDK on our
> production environment some time ago, but never restarted Artemis.
> Usually this doesn't produce any issues, but if we decide to restart
> Artemis after JDK upgrade it usually produces some errors about java
> binaries cannot be found. As if Artemis was still loaded on older JDK
> version and stopping it unloads these non-existing older JDK binaries from memory.
> Could it be that our Artemis instance is still using older JDK which
> exist only in memory, and console "inherits" that by trying to start
> itself, and then fails?
>
> Anyway, I see that JMX Remote is disabled by default. I have enabled
> JMX connector in management.xml, but I don't see port 1099 coming up.
> Is Artemis supposed to reload management.xml automatically, or do we
> have a chicken-egg problem now; cannot enable JMX because JMX is down?
>
> --
> Vilius
>
> -----Original Message-----
> From: Justin Bertram <jb...@apache.org>
> Sent: Wednesday, January 11, 2023 6:46 PM
> To: users@activemq.apache.org
> Subject: Re: reloading TLS certificate for Jolokia
>
> Those exceptions look internal to Jetty. I wouldn't expect them to
> necessarily be a problem for the restart process. Perhaps it was
> handling a request at the moment you tried to restart it.
>
> Do you ever see any of these messages in the log:
>
> Stopped embedded web server
>
> or
>
> Embedded web server not restarted in x milliseconds
>
> or
>
> Embedded web server restart failed
>
> If not, could you perhaps get a thread dump?
>
> In any event, if you want to start the embedded web server you can do
> so using JMX directly (e.g. via JConsole). Just invoke the
> startEmbeddedWebServer operation on the ActiveMQServerControl MBean.
>
>
> Justin
>
> On Wed, Jan 11, 2023 at 6:41 AM Vilius Šumskas
> <vi...@rivile.lt>
> wrote:
>
> > Well, I have found " restartEmbeddedWebServer()" and tried to use
> > that from the console, however now the console went down and doesn't come up.
> > The logs say: https://p.defau.lt/?C6ssFYW0JSJiZX1yszP0yQ
> >
> > Jolokia API is also down now. Is there a way to start embedded web
> > server without restarting the broker?
> >
> > --
> > Vilius
> >
> > -----Original Message-----
> > From: Vilius Šumskas <vi...@rivile.lt>
> > Sent: Wednesday, January 11, 2023 1:55 PM
> > To: users@activemq.apache.org
> > Subject: reloading TLS certificate for Jolokia
> >
> > Hi,
> >
> > since because of https://issues.apache.org/jira/browse/ARTEMIS-3117
> > ActiveMQ Artemis doesn‘t reload TLS certificates automatically
> > anymore I‘m wondering, what is the correct way to reload
> > certificates for embedded Jolokia Console?
> >
> > I have reloaded keystore for acceptors via console using reload() on
> > „artemis-ssl“ acceptor, but I just cannot figure out a way to reload
> > it for console itself.
> >
> > --
> > Best Regards,
> >
> > Vilius Šumskas
> > Rivile
> > IT manager
> >
> >
>
Re: reloading TLS certificate for Jolokia
Posted by Justin Bertram <jb...@apache.org>.
> Could it be that our Artemis instance is still using older JDK which
exist only in memory, and console "inherits" that by trying to start
itself, and then fails?
I don't have any experience with such a use-case so it's hard to say. The
"console" isn't fundamentally different from any other part of the broker.
It's all just Java classes. Therefore, I wouldn't expect such a failure to
be isolated there. If the rest of the broker is working fine I would expect
the embedded web server to work fine as well.
Are you _removing_ the version of Java that the broker is using during the
upgrade or are you just installing a new version alongside the existing
version? If the former, that seems dangerous in my opinion.
> Is Artemis supposed to reload management.xml automatically...
No. The contents of management.xml are not reloaded automatically.
> ...do we have a chicken-egg problem now; cannot enable JMX because JMX is
down?
You do have a chicken-egg problem with using JMX remotely. However, you
could still theoretically use it locally through a direct JVM connection.
You also still have the option of using management messages [1]. You can
find an example of their use in the examples/features/standard/management
directory shipped with the broker.
Justin
[1]
https://activemq.apache.org/components/artemis/documentation/latest/management.html#using-management-message-api
On Wed, Jan 11, 2023 at 2:49 PM Vilius Šumskas <vi...@rivile.lt>
wrote:
> No, I don't see any of these messages in the log. I have posted full log
> what I see.
>
> This is production system so getting threadump is not so easy, but I have
> decided to test this on our testing environment which runs newest 2.27.1.
> Running restartEmbeddedWebServer() via Hawtio console on that environment
> worked just fine.
>
> I have remembered one thing though. We have upgraded OpenJDK on our
> production environment some time ago, but never restarted Artemis. Usually
> this doesn't produce any issues, but if we decide to restart Artemis after
> JDK upgrade it usually produces some errors about java binaries cannot be
> found. As if Artemis was still loaded on older JDK version and stopping it
> unloads these non-existing older JDK binaries from memory.
> Could it be that our Artemis instance is still using older JDK which exist
> only in memory, and console "inherits" that by trying to start itself, and
> then fails?
>
> Anyway, I see that JMX Remote is disabled by default. I have enabled JMX
> connector in management.xml, but I don't see port 1099 coming up. Is
> Artemis supposed to reload management.xml automatically, or do we have a
> chicken-egg problem now; cannot enable JMX because JMX is down?
>
> --
> Vilius
>
> -----Original Message-----
> From: Justin Bertram <jb...@apache.org>
> Sent: Wednesday, January 11, 2023 6:46 PM
> To: users@activemq.apache.org
> Subject: Re: reloading TLS certificate for Jolokia
>
> Those exceptions look internal to Jetty. I wouldn't expect them to
> necessarily be a problem for the restart process. Perhaps it was handling a
> request at the moment you tried to restart it.
>
> Do you ever see any of these messages in the log:
>
> Stopped embedded web server
>
> or
>
> Embedded web server not restarted in x milliseconds
>
> or
>
> Embedded web server restart failed
>
> If not, could you perhaps get a thread dump?
>
> In any event, if you want to start the embedded web server you can do so
> using JMX directly (e.g. via JConsole). Just invoke the
> startEmbeddedWebServer operation on the ActiveMQServerControl MBean.
>
>
> Justin
>
> On Wed, Jan 11, 2023 at 6:41 AM Vilius Šumskas <vi...@rivile.lt>
> wrote:
>
> > Well, I have found " restartEmbeddedWebServer()" and tried to use that
> > from the console, however now the console went down and doesn't come up.
> > The logs say: https://p.defau.lt/?C6ssFYW0JSJiZX1yszP0yQ
> >
> > Jolokia API is also down now. Is there a way to start embedded web
> > server without restarting the broker?
> >
> > --
> > Vilius
> >
> > -----Original Message-----
> > From: Vilius Šumskas <vi...@rivile.lt>
> > Sent: Wednesday, January 11, 2023 1:55 PM
> > To: users@activemq.apache.org
> > Subject: reloading TLS certificate for Jolokia
> >
> > Hi,
> >
> > since because of https://issues.apache.org/jira/browse/ARTEMIS-3117
> > ActiveMQ Artemis doesn‘t reload TLS certificates automatically anymore
> > I‘m wondering, what is the correct way to reload certificates for
> > embedded Jolokia Console?
> >
> > I have reloaded keystore for acceptors via console using reload() on
> > „artemis-ssl“ acceptor, but I just cannot figure out a way to reload
> > it for console itself.
> >
> > --
> > Best Regards,
> >
> > Vilius Šumskas
> > Rivile
> > IT manager
> >
> >
>
RE: reloading TLS certificate for Jolokia
Posted by Vilius Šumskas <vi...@rivile.lt>.
No, I don't see any of these messages in the log. I have posted full log what I see.
This is production system so getting threadump is not so easy, but I have decided to test this on our testing environment which runs newest 2.27.1. Running restartEmbeddedWebServer() via Hawtio console on that environment worked just fine.
I have remembered one thing though. We have upgraded OpenJDK on our production environment some time ago, but never restarted Artemis. Usually this doesn't produce any issues, but if we decide to restart Artemis after JDK upgrade it usually produces some errors about java binaries cannot be found. As if Artemis was still loaded on older JDK version and stopping it unloads these non-existing older JDK binaries from memory.
Could it be that our Artemis instance is still using older JDK which exist only in memory, and console "inherits" that by trying to start itself, and then fails?
Anyway, I see that JMX Remote is disabled by default. I have enabled JMX connector in management.xml, but I don't see port 1099 coming up. Is Artemis supposed to reload management.xml automatically, or do we have a chicken-egg problem now; cannot enable JMX because JMX is down?
--
Vilius
-----Original Message-----
From: Justin Bertram <jb...@apache.org>
Sent: Wednesday, January 11, 2023 6:46 PM
To: users@activemq.apache.org
Subject: Re: reloading TLS certificate for Jolokia
Those exceptions look internal to Jetty. I wouldn't expect them to necessarily be a problem for the restart process. Perhaps it was handling a request at the moment you tried to restart it.
Do you ever see any of these messages in the log:
Stopped embedded web server
or
Embedded web server not restarted in x milliseconds
or
Embedded web server restart failed
If not, could you perhaps get a thread dump?
In any event, if you want to start the embedded web server you can do so using JMX directly (e.g. via JConsole). Just invoke the startEmbeddedWebServer operation on the ActiveMQServerControl MBean.
Justin
On Wed, Jan 11, 2023 at 6:41 AM Vilius Šumskas <vi...@rivile.lt>
wrote:
> Well, I have found " restartEmbeddedWebServer()" and tried to use that
> from the console, however now the console went down and doesn't come up.
> The logs say: https://p.defau.lt/?C6ssFYW0JSJiZX1yszP0yQ
>
> Jolokia API is also down now. Is there a way to start embedded web
> server without restarting the broker?
>
> --
> Vilius
>
> -----Original Message-----
> From: Vilius Šumskas <vi...@rivile.lt>
> Sent: Wednesday, January 11, 2023 1:55 PM
> To: users@activemq.apache.org
> Subject: reloading TLS certificate for Jolokia
>
> Hi,
>
> since because of https://issues.apache.org/jira/browse/ARTEMIS-3117
> ActiveMQ Artemis doesn‘t reload TLS certificates automatically anymore
> I‘m wondering, what is the correct way to reload certificates for
> embedded Jolokia Console?
>
> I have reloaded keystore for acceptors via console using reload() on
> „artemis-ssl“ acceptor, but I just cannot figure out a way to reload
> it for console itself.
>
> --
> Best Regards,
>
> Vilius Šumskas
> Rivile
> IT manager
>
>
Re: reloading TLS certificate for Jolokia
Posted by Justin Bertram <jb...@apache.org>.
Those exceptions look internal to Jetty. I wouldn't expect them to
necessarily be a problem for the restart process. Perhaps it was handling a
request at the moment you tried to restart it.
Do you ever see any of these messages in the log:
Stopped embedded web server
or
Embedded web server not restarted in x milliseconds
or
Embedded web server restart failed
If not, could you perhaps get a thread dump?
In any event, if you want to start the embedded web server you can do so
using JMX directly (e.g. via JConsole). Just invoke the
startEmbeddedWebServer operation on the ActiveMQServerControl MBean.
Justin
On Wed, Jan 11, 2023 at 6:41 AM Vilius Šumskas <vi...@rivile.lt>
wrote:
> Well, I have found " restartEmbeddedWebServer()" and tried to use that
> from the console, however now the console went down and doesn't come up.
> The logs say: https://p.defau.lt/?C6ssFYW0JSJiZX1yszP0yQ
>
> Jolokia API is also down now. Is there a way to start embedded web server
> without restarting the broker?
>
> --
> Vilius
>
> -----Original Message-----
> From: Vilius Šumskas <vi...@rivile.lt>
> Sent: Wednesday, January 11, 2023 1:55 PM
> To: users@activemq.apache.org
> Subject: reloading TLS certificate for Jolokia
>
> Hi,
>
> since because of https://issues.apache.org/jira/browse/ARTEMIS-3117
> ActiveMQ Artemis doesn‘t reload TLS certificates automatically anymore I‘m
> wondering, what is the correct way to reload certificates for embedded
> Jolokia Console?
>
> I have reloaded keystore for acceptors via console using reload() on
> „artemis-ssl“ acceptor, but I just cannot figure out a way to reload it for
> console itself.
>
> --
> Best Regards,
>
> Vilius Šumskas
> Rivile
> IT manager
>
>
RE: reloading TLS certificate for Jolokia
Posted by Vilius Šumskas <vi...@rivile.lt>.
Well, I have found " restartEmbeddedWebServer()" and tried to use that from the console, however now the console went down and doesn't come up. The logs say: https://p.defau.lt/?C6ssFYW0JSJiZX1yszP0yQ
Jolokia API is also down now. Is there a way to start embedded web server without restarting the broker?
--
Vilius
-----Original Message-----
From: Vilius Šumskas <vi...@rivile.lt>
Sent: Wednesday, January 11, 2023 1:55 PM
To: users@activemq.apache.org
Subject: reloading TLS certificate for Jolokia
Hi,
since because of https://issues.apache.org/jira/browse/ARTEMIS-3117 ActiveMQ Artemis doesn‘t reload TLS certificates automatically anymore I‘m wondering, what is the correct way to reload certificates for embedded Jolokia Console?
I have reloaded keystore for acceptors via console using reload() on „artemis-ssl“ acceptor, but I just cannot figure out a way to reload it for console itself.
--
Best Regards,
Vilius Šumskas
Rivile
IT manager