You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficcontrol.apache.org by Jason Tucker <ja...@gmail.com> on 2017/07/11 21:37:01 UTC
traffic_router tomcat init script (shutdown) problems
FYI - opened ticket and PR for this issue:
The tomcat init script has a few problems:
1. "Clean" shutdowns frequently timeout, and the scripts give up, leaving
tomcat running
2. Normal tomcat shutdown actually involves spinning up a second jvm
instance. Right now, we start this second instance with the same
CATALINA_OPTS as traffic_router, which can be problematic on
memory-constrained hosts.
https://issues.apache.org/jira/browse/TC-416
https://github.com/apache/incubator-trafficcontrol/pull/724
Thanks,
_Jason
Re: traffic_router tomcat init script (shutdown) problems
Posted by Jason Tucker <ja...@gmail.com>.
I don't know if we've experienced problems with stopping traffic_monitor in
the past, but it certainly has the potential to have the same problem. The
only real difference between those two files was the CATALINA_OPTS variable
values (and also license header?)
I have an updated script for traffic_monitor here:
https://github.com/guzzijason/incubator-trafficcontrol/commit/7631f81ea754bbe0f63ac77120df5686c2310dd6
I can PR that if folks feel there is a need.
__Jason
On Wed, Jul 12, 2017 at 1:39 PM, Steve Malenfant <sm...@gmail.com>
wrote:
> Does Traffic Monitor (Java version) have the same problem? If it does, can
> the fix be applied?
>
> On Wed, Jul 12, 2017 at 12:43 PM, Jason Tucker <ja...@gmail.com>
> wrote:
>
>> Thanks, Jeff.
>>
>> And just to clarify - this will *try* to do a graceful shutdown first,
>> wait
>> for the timeout period (10 seconds) and then force it if that was
>> unsuccessful. The previous behavior was to wait for 5 seconds, and then
>> give up entirely.
>>
>> We'll have to take another look at this when we move to a newer tomcat
>> package. I have some ideas on how to improve this further - like switching
>> to a systemd unit rather than init script for el7, and putting tunables in
>> an external config (i.e. /etc/sysconfig/tomcat) like the tomcat RPMs
>> generally do.
>>
>> __Jason
>>
>> On Wed, Jul 12, 2017 at 11:44 AM, Jeff Elsloo <el...@apache.org> wrote:
>>
>> > I discussed this with Jason, reviewed the PR and will be merging it
>> > soon unless someone has concerns. I asked specifically about "force"
>> > being the default shutdown mode, and that was done intentionally.
>> > There might be a use case for a graceful shutdown with typical
>> > applications deployed into Tomcat, but Traffic Router does not service
>> > any long running sessions, so getting it shut down quickly is actually
>> > desired.
>> >
>> > We can use this new init script and make changes as necessary in the
>> > future, but this should be an improvement. Hopefully we won't have to
>> > `kill -9 <pid>` anymore.
>> > --
>> > Thanks,
>> > Jeff
>> >
>> >
>> > On Tue, Jul 11, 2017 at 3:37 PM, Jason Tucker <ja...@gmail.com>
>> > wrote:
>> > > FYI - opened ticket and PR for this issue:
>> > >
>> > > The tomcat init script has a few problems:
>> > >
>> > > 1. "Clean" shutdowns frequently timeout, and the scripts give up,
>> leaving
>> > > tomcat running
>> > >
>> > > 2. Normal tomcat shutdown actually involves spinning up a second jvm
>> > > instance. Right now, we start this second instance with the same
>> > > CATALINA_OPTS as traffic_router, which can be problematic on
>> > > memory-constrained hosts.
>> > >
>> > > https://issues.apache.org/jira/browse/TC-416
>> > > https://github.com/apache/incubator-trafficcontrol/pull/724
>> > >
>> > > Thanks,
>> > >
>> > > _Jason
>> >
>>
>
>
Re: traffic_router tomcat init script (shutdown) problems
Posted by Steve Malenfant <sm...@gmail.com>.
Does Traffic Monitor (Java version) have the same problem? If it does, can
the fix be applied?
On Wed, Jul 12, 2017 at 12:43 PM, Jason Tucker <ja...@gmail.com>
wrote:
> Thanks, Jeff.
>
> And just to clarify - this will *try* to do a graceful shutdown first, wait
> for the timeout period (10 seconds) and then force it if that was
> unsuccessful. The previous behavior was to wait for 5 seconds, and then
> give up entirely.
>
> We'll have to take another look at this when we move to a newer tomcat
> package. I have some ideas on how to improve this further - like switching
> to a systemd unit rather than init script for el7, and putting tunables in
> an external config (i.e. /etc/sysconfig/tomcat) like the tomcat RPMs
> generally do.
>
> __Jason
>
> On Wed, Jul 12, 2017 at 11:44 AM, Jeff Elsloo <el...@apache.org> wrote:
>
> > I discussed this with Jason, reviewed the PR and will be merging it
> > soon unless someone has concerns. I asked specifically about "force"
> > being the default shutdown mode, and that was done intentionally.
> > There might be a use case for a graceful shutdown with typical
> > applications deployed into Tomcat, but Traffic Router does not service
> > any long running sessions, so getting it shut down quickly is actually
> > desired.
> >
> > We can use this new init script and make changes as necessary in the
> > future, but this should be an improvement. Hopefully we won't have to
> > `kill -9 <pid>` anymore.
> > --
> > Thanks,
> > Jeff
> >
> >
> > On Tue, Jul 11, 2017 at 3:37 PM, Jason Tucker <ja...@gmail.com>
> > wrote:
> > > FYI - opened ticket and PR for this issue:
> > >
> > > The tomcat init script has a few problems:
> > >
> > > 1. "Clean" shutdowns frequently timeout, and the scripts give up,
> leaving
> > > tomcat running
> > >
> > > 2. Normal tomcat shutdown actually involves spinning up a second jvm
> > > instance. Right now, we start this second instance with the same
> > > CATALINA_OPTS as traffic_router, which can be problematic on
> > > memory-constrained hosts.
> > >
> > > https://issues.apache.org/jira/browse/TC-416
> > > https://github.com/apache/incubator-trafficcontrol/pull/724
> > >
> > > Thanks,
> > >
> > > _Jason
> >
>
Re: traffic_router tomcat init script (shutdown) problems
Posted by Jason Tucker <ja...@gmail.com>.
Thanks, Jeff.
And just to clarify - this will *try* to do a graceful shutdown first, wait
for the timeout period (10 seconds) and then force it if that was
unsuccessful. The previous behavior was to wait for 5 seconds, and then
give up entirely.
We'll have to take another look at this when we move to a newer tomcat
package. I have some ideas on how to improve this further - like switching
to a systemd unit rather than init script for el7, and putting tunables in
an external config (i.e. /etc/sysconfig/tomcat) like the tomcat RPMs
generally do.
__Jason
On Wed, Jul 12, 2017 at 11:44 AM, Jeff Elsloo <el...@apache.org> wrote:
> I discussed this with Jason, reviewed the PR and will be merging it
> soon unless someone has concerns. I asked specifically about "force"
> being the default shutdown mode, and that was done intentionally.
> There might be a use case for a graceful shutdown with typical
> applications deployed into Tomcat, but Traffic Router does not service
> any long running sessions, so getting it shut down quickly is actually
> desired.
>
> We can use this new init script and make changes as necessary in the
> future, but this should be an improvement. Hopefully we won't have to
> `kill -9 <pid>` anymore.
> --
> Thanks,
> Jeff
>
>
> On Tue, Jul 11, 2017 at 3:37 PM, Jason Tucker <ja...@gmail.com>
> wrote:
> > FYI - opened ticket and PR for this issue:
> >
> > The tomcat init script has a few problems:
> >
> > 1. "Clean" shutdowns frequently timeout, and the scripts give up, leaving
> > tomcat running
> >
> > 2. Normal tomcat shutdown actually involves spinning up a second jvm
> > instance. Right now, we start this second instance with the same
> > CATALINA_OPTS as traffic_router, which can be problematic on
> > memory-constrained hosts.
> >
> > https://issues.apache.org/jira/browse/TC-416
> > https://github.com/apache/incubator-trafficcontrol/pull/724
> >
> > Thanks,
> >
> > _Jason
>
Re: traffic_router tomcat init script (shutdown) problems
Posted by Jeff Elsloo <el...@apache.org>.
I discussed this with Jason, reviewed the PR and will be merging it
soon unless someone has concerns. I asked specifically about "force"
being the default shutdown mode, and that was done intentionally.
There might be a use case for a graceful shutdown with typical
applications deployed into Tomcat, but Traffic Router does not service
any long running sessions, so getting it shut down quickly is actually
desired.
We can use this new init script and make changes as necessary in the
future, but this should be an improvement. Hopefully we won't have to
`kill -9 <pid>` anymore.
--
Thanks,
Jeff
On Tue, Jul 11, 2017 at 3:37 PM, Jason Tucker <ja...@gmail.com> wrote:
> FYI - opened ticket and PR for this issue:
>
> The tomcat init script has a few problems:
>
> 1. "Clean" shutdowns frequently timeout, and the scripts give up, leaving
> tomcat running
>
> 2. Normal tomcat shutdown actually involves spinning up a second jvm
> instance. Right now, we start this second instance with the same
> CATALINA_OPTS as traffic_router, which can be problematic on
> memory-constrained hosts.
>
> https://issues.apache.org/jira/browse/TC-416
> https://github.com/apache/incubator-trafficcontrol/pull/724
>
> Thanks,
>
> _Jason