You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Jan Tosovsky <j....@email.cz> on 2022/04/11 10:17:54 UTC
Tomcat service switches to another JDK under the hood
We have a mixed JDK environment on our internal Windows Server.
Tomcat is installed using the service installer and its service
configuration points to the JDK 17 as it serves some internal apps requiring
JDK 12+.
JAVA_HOME points to older JDK 8.
PATH variable contains link to JDK 8, but also to JDK 11 in this order (the
latter link is a bug I spotted right now).
And now weirdness comes. From time to time the tomcat somehow switches to
older JDK: we detect this by the fact the internal webapp complains about
the issue only present in pre-JDK 12 versions. I assume JDK 11 is used as
this is related to JDK HttpClient which is not the part of the JDK 8.
It is strange the only reference to JDK 11 is that PATH entry, but is the
second one so JDK 8 should win if the PATH is somehow involved.
Actual case: the tomcat was restarted manually on Friday using tray icon and
in the log I can see that JDK 17 was picked up. There was no user activity
during the weekend. Today when accessing the app error is raised. Manual
restart via try icon fixed the issue.
This issue is not urgent in any case, there are no mission-critical apps
running on our server. But I am quite surprised by this behavior. Kind of
the race condition, which could be dangerous.
Regards, Jan
RE: Tomcat service switches to another JDK under the hood
Posted by Jan Tosovsky <j....@email.cz>.
I have some updates.
First of all, there is no JDK switch under the hood. I am glad my conclusion was wrong.
In my REST API client app I use Java 11+ HTTPClient. For the POST call the external API requires Content-Lenght header even the body is empty, which HTTPClient doesn't support out of the box. You can add this header manually, but it is only allowed in JDK12+ together with the setting the system property jdk.httpclient.allowRestrictedHeaders=content-length. This property has to be set before instantiating the HTTPClient, so I set the system property directly in the code the line before httpClient initialization.
It worked fine after deployment, but stopped working after while (next day). Now it is clear that even setting the system property in the code, it is ignored by that httpClient initialization code. It sounds like HTTPClient instance is somehow reused/shared across threads/apps and some created without this system property in place was picked up.
Now I am setting the system property as the command line param when starting the tomcat and issue has gone (= all httpClient instances are equivalent).
Jan
-----Original Message-----
From: Zdeněk Henek <vr...@gmail.com>
Sent: Monday, April 11, 2022 2:35 PM
To: Tomcat Users List <us...@tomcat.apache.org>
Subject: Re: Tomcat service switches to another JDK under the hood
Nothing you mentioned could have any effect.
How do you ensure jdks are not mixing between tomcat and the other jdk8 application?
Try to use separated cmd files as described above for both Tomcat and the other application and remove all jdk configurations from windows variables just to test this option. I think it is much better to keep all config separated especially when you have two apps each using different jdk.
Regards,
Zdenek Henek
On Mon, Apr 11, 2022 at 1:50 PM Jan Tosovsky <j....@email.cz> wrote:
> There isn't any sign of the tomcat restart in the tomcat logs. In
> tomcat9-stderr-2022-04-08.log there is just one bootstrap section with
> the path to JDK 17, followed by mesages about the registering webapps.
> Then weekend of silence interrupted by today's exception.
>
> In that bootstrap section there are listed exact params I can see in
> the service configuration accessible via tray icon. So in this stage I
> believe tomcat does its job properly. There is no interference with OS settings.
> But then something breaks.
>
>
> If this could have some infuence:
> - one of tomcat webapps is based on Quartz scheduler running various
> tasks including java based utilities (i.e. running their own JVMs)
> - our apps are served by Apache HTTP via AJP
> - all webapps are JSF based, no DB, just simple controller; the main
> purpose is to process user input or fetch external REST API services
> and display the result
> - there is one extra 3rd party "hotfolder" Java app running as a
> service (JDK 8 is used in this case)
>
>
> -----Original Message-----
> From: Zdeněk Henek <vr...@gmail.com>
> Sent: Monday, April 11, 2022 1:04 PM
> To: Tomcat Users List <us...@tomcat.apache.org>
> Subject: Re: Tomcat service switches to another JDK under the hood
>
> Hi Jan,
>
> looks like somebody/something has restarted tomcat with different
> JAVA_HOME/PATH variables. It is not possible to change jvm process
> while running.
>
> Another option could be you have there running two tomcat instances.
>
> Create cmd file e.g. tomcatStart.cmd with content like
>
> set JAVA_HOME=c:/...
> set CATALINA_HOME=c:/......
> set PATH=%JAVA_HOME%/bin;%PATH%
>
> and use only this file to start tomcat. Put there other variables eg.
> JAVA_OPTS in case you have some other jvm variables.
> Do not rely on MS Windows variables settings.
>
> Regards,
>
> Zdenek Henek
>
>
> On Mon, Apr 11, 2022 at 12:18 PM Jan Tosovsky <j....@email.cz> wrote:
>
> > We have a mixed JDK environment on our internal Windows Server.
> >
> > Tomcat is installed using the service installer and its service
> > configuration points to the JDK 17 as it serves some internal apps
> > requiring JDK 12+.
> >
> >
> >
> > JAVA_HOME points to older JDK 8.
> >
> > PATH variable contains link to JDK 8, but also to JDK 11 in this
> > order (the latter link is a bug I spotted right now).
> >
> >
> >
> > And now weirdness comes. From time to time the tomcat somehow
> > switches to older JDK: we detect this by the fact the internal
> > webapp complains about the issue only present in pre-JDK 12
> > versions. I assume JDK 11 is used as this is related to JDK
> > HttpClient which is not the part of
> the JDK 8.
> >
> > It is strange the only reference to JDK 11 is that PATH entry, but
> > is the second one so JDK 8 should win if the PATH is somehow involved.
> >
> >
> >
> > Actual case: the tomcat was restarted manually on Friday using tray
> > icon and in the log I can see that JDK 17 was picked up. There was
> > no user activity during the weekend. Today when accessing the app
> > error is raised. Manual restart via try icon fixed the issue.
> >
> >
> >
> > This issue is not urgent in any case, there are no mission-critical
> > apps running on our server. But I am quite surprised by this behavior.
> > Kind of the race condition, which could be dangerous.
> >
> >
> >
> > Regards, Jan
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat service switches to another JDK under the hood
Posted by Zdeněk Henek <vr...@gmail.com>.
Nothing you mentioned could have any effect.
How do you ensure jdks are not mixing between tomcat and the other jdk8
application?
Try to use separated cmd files as described above for both Tomcat and the
other application and remove all jdk configurations from windows variables
just to test this option. I think it is much better to keep all config
separated especially when you have two apps each using different jdk.
Regards,
Zdenek Henek
On Mon, Apr 11, 2022 at 1:50 PM Jan Tosovsky <j....@email.cz> wrote:
> There isn't any sign of the tomcat restart in the tomcat logs. In
> tomcat9-stderr-2022-04-08.log there is just one bootstrap section with the
> path to JDK 17, followed by mesages about the registering webapps. Then
> weekend of silence interrupted by today's exception.
>
> In that bootstrap section there are listed exact params I can see in the
> service configuration accessible via tray icon. So in this stage I believe
> tomcat does its job properly. There is no interference with OS settings.
> But then something breaks.
>
>
> If this could have some infuence:
> - one of tomcat webapps is based on Quartz scheduler running various tasks
> including java based utilities (i.e. running their own JVMs)
> - our apps are served by Apache HTTP via AJP
> - all webapps are JSF based, no DB, just simple controller; the main
> purpose is to process user input or fetch external REST API services and
> display the result
> - there is one extra 3rd party "hotfolder" Java app running as a service
> (JDK 8 is used in this case)
>
>
> -----Original Message-----
> From: Zdeněk Henek <vr...@gmail.com>
> Sent: Monday, April 11, 2022 1:04 PM
> To: Tomcat Users List <us...@tomcat.apache.org>
> Subject: Re: Tomcat service switches to another JDK under the hood
>
> Hi Jan,
>
> looks like somebody/something has restarted tomcat with different
> JAVA_HOME/PATH variables. It is not possible to change jvm process while
> running.
>
> Another option could be you have there running two tomcat instances.
>
> Create cmd file e.g. tomcatStart.cmd with content like
>
> set JAVA_HOME=c:/...
> set CATALINA_HOME=c:/......
> set PATH=%JAVA_HOME%/bin;%PATH%
>
> and use only this file to start tomcat. Put there other variables eg.
> JAVA_OPTS in case you have some other jvm variables.
> Do not rely on MS Windows variables settings.
>
> Regards,
>
> Zdenek Henek
>
>
> On Mon, Apr 11, 2022 at 12:18 PM Jan Tosovsky <j....@email.cz> wrote:
>
> > We have a mixed JDK environment on our internal Windows Server.
> >
> > Tomcat is installed using the service installer and its service
> > configuration points to the JDK 17 as it serves some internal apps
> > requiring JDK 12+.
> >
> >
> >
> > JAVA_HOME points to older JDK 8.
> >
> > PATH variable contains link to JDK 8, but also to JDK 11 in this order
> > (the latter link is a bug I spotted right now).
> >
> >
> >
> > And now weirdness comes. From time to time the tomcat somehow switches
> > to older JDK: we detect this by the fact the internal webapp complains
> > about the issue only present in pre-JDK 12 versions. I assume JDK 11
> > is used as this is related to JDK HttpClient which is not the part of
> the JDK 8.
> >
> > It is strange the only reference to JDK 11 is that PATH entry, but is
> > the second one so JDK 8 should win if the PATH is somehow involved.
> >
> >
> >
> > Actual case: the tomcat was restarted manually on Friday using tray
> > icon and in the log I can see that JDK 17 was picked up. There was no
> > user activity during the weekend. Today when accessing the app error
> > is raised. Manual restart via try icon fixed the issue.
> >
> >
> >
> > This issue is not urgent in any case, there are no mission-critical
> > apps running on our server. But I am quite surprised by this behavior.
> > Kind of the race condition, which could be dangerous.
> >
> >
> >
> > Regards, Jan
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
RE: Tomcat service switches to another JDK under the hood
Posted by Jan Tosovsky <j....@email.cz>.
There isn't any sign of the tomcat restart in the tomcat logs. In tomcat9-stderr-2022-04-08.log there is just one bootstrap section with the path to JDK 17, followed by mesages about the registering webapps. Then weekend of silence interrupted by today's exception.
In that bootstrap section there are listed exact params I can see in the service configuration accessible via tray icon. So in this stage I believe tomcat does its job properly. There is no interference with OS settings. But then something breaks.
If this could have some infuence:
- one of tomcat webapps is based on Quartz scheduler running various tasks including java based utilities (i.e. running their own JVMs)
- our apps are served by Apache HTTP via AJP
- all webapps are JSF based, no DB, just simple controller; the main purpose is to process user input or fetch external REST API services and display the result
- there is one extra 3rd party "hotfolder" Java app running as a service (JDK 8 is used in this case)
-----Original Message-----
From: Zdeněk Henek <vr...@gmail.com>
Sent: Monday, April 11, 2022 1:04 PM
To: Tomcat Users List <us...@tomcat.apache.org>
Subject: Re: Tomcat service switches to another JDK under the hood
Hi Jan,
looks like somebody/something has restarted tomcat with different JAVA_HOME/PATH variables. It is not possible to change jvm process while running.
Another option could be you have there running two tomcat instances.
Create cmd file e.g. tomcatStart.cmd with content like
set JAVA_HOME=c:/...
set CATALINA_HOME=c:/......
set PATH=%JAVA_HOME%/bin;%PATH%
and use only this file to start tomcat. Put there other variables eg.
JAVA_OPTS in case you have some other jvm variables.
Do not rely on MS Windows variables settings.
Regards,
Zdenek Henek
On Mon, Apr 11, 2022 at 12:18 PM Jan Tosovsky <j....@email.cz> wrote:
> We have a mixed JDK environment on our internal Windows Server.
>
> Tomcat is installed using the service installer and its service
> configuration points to the JDK 17 as it serves some internal apps
> requiring JDK 12+.
>
>
>
> JAVA_HOME points to older JDK 8.
>
> PATH variable contains link to JDK 8, but also to JDK 11 in this order
> (the latter link is a bug I spotted right now).
>
>
>
> And now weirdness comes. From time to time the tomcat somehow switches
> to older JDK: we detect this by the fact the internal webapp complains
> about the issue only present in pre-JDK 12 versions. I assume JDK 11
> is used as this is related to JDK HttpClient which is not the part of the JDK 8.
>
> It is strange the only reference to JDK 11 is that PATH entry, but is
> the second one so JDK 8 should win if the PATH is somehow involved.
>
>
>
> Actual case: the tomcat was restarted manually on Friday using tray
> icon and in the log I can see that JDK 17 was picked up. There was no
> user activity during the weekend. Today when accessing the app error
> is raised. Manual restart via try icon fixed the issue.
>
>
>
> This issue is not urgent in any case, there are no mission-critical
> apps running on our server. But I am quite surprised by this behavior.
> Kind of the race condition, which could be dangerous.
>
>
>
> Regards, Jan
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat service switches to another JDK under the hood
Posted by Zdeněk Henek <vr...@gmail.com>.
Hi Jan,
looks like somebody/something has restarted tomcat with different
JAVA_HOME/PATH variables. It is not possible to change jvm process while
running.
Another option could be you have there running two tomcat instances.
Create cmd file e.g. tomcatStart.cmd with content like
set JAVA_HOME=c:/...
set CATALINA_HOME=c:/......
set PATH=%JAVA_HOME%/bin;%PATH%
and use only this file to start tomcat. Put there other variables eg.
JAVA_OPTS in case you have some other jvm variables.
Do not rely on MS Windows variables settings.
Regards,
Zdenek Henek
On Mon, Apr 11, 2022 at 12:18 PM Jan Tosovsky <j....@email.cz> wrote:
> We have a mixed JDK environment on our internal Windows Server.
>
> Tomcat is installed using the service installer and its service
> configuration points to the JDK 17 as it serves some internal apps
> requiring
> JDK 12+.
>
>
>
> JAVA_HOME points to older JDK 8.
>
> PATH variable contains link to JDK 8, but also to JDK 11 in this order (the
> latter link is a bug I spotted right now).
>
>
>
> And now weirdness comes. From time to time the tomcat somehow switches to
> older JDK: we detect this by the fact the internal webapp complains about
> the issue only present in pre-JDK 12 versions. I assume JDK 11 is used as
> this is related to JDK HttpClient which is not the part of the JDK 8.
>
> It is strange the only reference to JDK 11 is that PATH entry, but is the
> second one so JDK 8 should win if the PATH is somehow involved.
>
>
>
> Actual case: the tomcat was restarted manually on Friday using tray icon
> and
> in the log I can see that JDK 17 was picked up. There was no user activity
> during the weekend. Today when accessing the app error is raised. Manual
> restart via try icon fixed the issue.
>
>
>
> This issue is not urgent in any case, there are no mission-critical apps
> running on our server. But I am quite surprised by this behavior. Kind of
> the race condition, which could be dangerous.
>
>
>
> Regards, Jan
>
>
>
>