You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by "Edward Hart (c)" <eh...@vmware.com> on 2016/02/11 20:20:14 UTC

Tomcat Failure Modes

Q1:  Can Tomcat be configured to 'roll back' if a patch update causes a TC failure on a production server?
Q2:  Can TC be configured to fail to a known safe state in the event of server failure during operation?

I am developing a Security Technical Implement Guide (STIG) for Tomcat.  A STIG is essentially a detailed checklist for hardening a given technology.  DoD uses them to provide cyber defense.

Finding configurable ways to satisfy the below 2 requirements is proving difficult.

Req 1 : The web server must augment re-creation to a stable and known baseline.

Req 2 : The web server must be built to fail to a known safe state if system initialization fails, shutdown fails, or aborts fail.


Req 1 is intended to provide a means to roll back to a last known stable environment in case a patch fails.  Req 2 is intended to provide fail safe environments in case something (perhaps an attacker) causes system failure.

Thanks,

Ed


Re: Tomcat Failure Modes

Posted by David kerber <dc...@verizon.net>.
On 2/11/2016 2:20 PM, Edward Hart (c) wrote:
> Q1:  Can Tomcat be configured to 'roll back' if a patch update causes a TC failure on a production server?
> Q2:  Can TC be configured to fail to a known safe state in the event of server failure during operation?
>
> I am developing a Security Technical Implement Guide (STIG) for Tomcat.  A STIG is essentially a detailed checklist for hardening a given technology.  DoD uses them to provide cyber defense.
>
> Finding configurable ways to satisfy the below 2 requirements is proving difficult.
>
> Req 1 : The web server must augment re-creation to a stable and known baseline.
>
> Req 2 : The web server must be built to fail to a known safe state if system initialization fails, shutdown fails, or aborts fail.
>
>
> Req 1 is intended to provide a means to roll back to a last known stable environment in case a patch fails.  Req 2 is intended to provide fail safe environments in case something (perhaps an attacker) causes system failure.

#1 is probably doable with some kind of version control.

#2 seems to be to be most easily answered by staying shut down as its 
known stable state.



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat Failure Modes

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mark,

On 2/11/16 2:46 PM, Mark Thomas wrote:
> On 11/02/2016 19:20, Edward Hart (c) wrote:
>> Q1:  Can Tomcat be configured to 'roll back' if a patch update
>> causes a TC failure on a production server?
> 
> The question makes no sense since there is no such thing as a
> patch update for Tomcat.

+1

Besides, if Tomcat was failing and it could recover in any way, then
it wouldn't be failing, now would it?

(I love these crazy defense requirements: guaranteed absolutely no
failures ever. Oh, and by the way, when the failure occurs, it must be
self-healing. Bwahahahahahaha.)

>> Q2:  Can TC be configured to fail to a known safe state in the
>> event of server failure during operation?
> 
> Define "server failure". Define "safe state".

I don't think it matters what these definitions are. The answer is
"no" and the right question is "can the server be confused to failto a
known safe state" and that answer is "yes": just use a virtual machine
with snapshot-and rollback capabilities or, better yet, use a
deployment-management package (e.g. docker) so that anything that
fails some kind of probing test can be automatically decommissioned
and replaced with a new server built with a "known safe state".

> Generally if Tomcat ever entered an "unsafe state" that would be a 
> security vulnerability.
> 
>> I am developing a Security Technical Implement Guide (STIG) for
>> Tomcat.  A STIG is essentially a detailed checklist for hardening
>> a given technology.  DoD uses them to provide cyber defense.
>> 
>> Finding configurable ways to satisfy the below 2 requirements is
>> proving difficult.
>> 
>> Req 1 : The web server must augment re-creation to a stable and
>> known baseline.

What in the world does "augment re-creation to a stable and known
state" mean?

>> Req 2 : The web server must be built to fail to a known safe
>> state if system initialization fails, shutdown fails, or aborts
>> fail.
> 
> Define "an abort"

And what happens when "an abort" fails? Abort, Retry, Fail?

>> Req 1 is intended to provide a means to roll back to a last
>> known stable environment in case a patch fails.
> 
> One could argue that is met by using separate $CATALINA_HOME and 
> $CATALINA_BASE (see RUINNING.txt).

This is not automatic. Unless you mean "policy is to have a human
being automatically revert the deployment to using the
previously-vetted working version of Tomcat".

> One could also argue that since all configuration is in the 
> $CATALINA_BASE/conf directory, integration with the DoD
> configuration management system of choice should be trivial

Ideally. That depends upon how stupid the DoD configuration management
system is.

>> Req 2 is intended to provide fail safe environments in case 
>> something (perhaps an attacker) causes system failure.
> 
> See comment above regarding unsafe states.

Yep. Automating fail-safes is insanely difficult to do. This is not a
problem with Tomcat, Java, OSS, etc. It's a fact of human-built
systems. There are only a certain number of ways you can predict
something will fail, and a certain number of ways of detecting those
failures. Any oversight in those two areas means that the game is over
and you can have a system in an unsafe state that you don't know about.

The best you can do is attempt to detect crazy behavior and then kill
the server when that kind of thing happens. But be aware that killing
one server due to (apparently) crazy behavior may cause crazy-looking
behavior on the remaining servers (e.g. a spike in resource-usage,
connections, etc.) due to fail-over. Make sure your system doesn't
cause a cascade failure to itself by attempting to "fail safe".

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAla98coACgkQ9CaO5/Lv0PCNkACgvizc0PEXshw1QBbE3GuZAVZA
aSAAn1E3ICGQQK0WLRcEsGRyZ1J0hEpp
=eu3S
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat Failure Modes

Posted by Mark Thomas <ma...@apache.org>.
On 11/02/2016 19:20, Edward Hart (c) wrote:
> Q1:  Can Tomcat be configured to 'roll back' if a patch update causes a TC failure on a production server?

The question makes no sense since there is no such thing as a patch
update for Tomcat.

> Q2:  Can TC be configured to fail to a known safe state in the event of server failure during operation?

Define "server failure". Define "safe state".

Generally if Tomcat ever entered an "unsafe state" that would be a
security vulnerability.

> I am developing a Security Technical Implement Guide (STIG) for Tomcat.  A STIG is essentially a detailed checklist for hardening a given technology.  DoD uses them to provide cyber defense.
> 
> Finding configurable ways to satisfy the below 2 requirements is proving difficult.
> 
> Req 1 : The web server must augment re-creation to a stable and known baseline.
> 
> Req 2 : The web server must be built to fail to a known safe state if system initialization fails, shutdown fails, or aborts fail.

Define "an abort"

> Req 1 is intended to provide a means to roll back to a last known stable environment in case a patch fails.

One could argue that is met by using separate $CATALINA_HOME and
$CATALINA_BASE (see RUINNING.txt).

One could also argue that since all configuration is in the
$CATALINA_BASE/conf directory, integration with the DoD configuration
management system of choice should be trivial

> Req 2 is intended to provide fail safe environments in case something (perhaps an attacker) causes system failure.

See comment above regarding unsafe states.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org