CIS Tomcat 8 Benchmark (v1.1.0) -- Questions

[Robert Turner <rt...@e-djuster.ca>]

While I think I know the answer to my question, I wanted to double-check
with the group to confirm.

I have been asked to perform the CIS Apache Tomcat 8 Benchmark (v1.1.0) on
our production Tomcat installation, and I am looking through the questions
/ information extraction requests, and I suspect they are not really
evaluating what they think they are, and furthermore encouraging bad
practices.

For instance, the first entry I have in the spreadsheet I was provided is
listed as follows:

CIS Control:
2.1 Alter the Advertised server.info String (Scored)

Description:
The server.info attribute contains the name of the application service.
This value is presented to Tomcat clients when clients connect to the
tomcat server.

Audit Procedures:
Perform the following to determine if the server.info value has been
changed:
Extract the ServerInfo.properties file and examine the server.info
attribute.
$ cd $CATALINA_HOME/lib
$ jar xf catalina.jar org/apache/catalina/util/ServerInfo.properties
$ grep server.info org/apache/catalina/util/ServerInfo.properties


So, other than a few issues with the audit procedures, etc. This seems to
be doing the following:

a) evaluating a default value which I believe can be overridden and thus
may not actually reflect the value the server may provide to external
clients
b) encouraging the modification of the catalina.jar contents to correct the
default value

There are a few similar items (for server.number, server.built) (2.2, 2.3).


Thoughts / comments from "those in the know"?

Thanks,

Robert

Re: CIS Tomcat 8 Benchmark (v1.1.0) -- Questions

[Robert Turner <rt...@e-djuster.ca>]

Thanks Mark -- I appreciate your insights, and I agree with not donating
your time so someone else can make money off it. It definitely seems
somewhat unethical on their part not to make those resources available for
free if they are sourcing them for free. I would imagine it's still easy
enough to build up a commercial service around it, even if the artifacts
are all public.

I have traditionally relied on the advice in the Tomcat documentation in
precedence over anything the auditors "recommend" for the reasons you
articulated. Unfortunately, "audit" has a "special place" in the world ...

We do already override the "Server" header which is the obvious visible
version information (as so many "audits" insist on this being a "good
thing"), but I agree about staying on top of versions as the more correct
solution (rant: if only AWS would keep their Elastic Beanstalk platform up
to date as well...never mind that they skipped version 9, making it quite
awkward to jump from 8.5 to 10). Thanks for the additional details on that
one should it come up as a "finding".

I was only using that one as a "example", more to start a discussion, and
to ensure I wasn't "out to lunch" on my thoughts. There are plenty of
others entries in the evaluation document where the method for determining
is too narrow, and relies on a specific version (often old) or a particular
location for configuration (like explicitly specifying options in
catalina.sh, even though there are more appropriate places, like in
$CATALINA_HOME/conf files, or in the applications), as well as some items
that are somewhat obscure and don't result in improving security in any
material way (i.e. obscuring, rather than securing).

Robert


On Tue, Sep 5, 2023 at 11:27 AM Mark Thomas <ma...@apache.org> wrote:

> I spoke to some CIS representatives at a recent conference given that I
> have concerns about the quality of some of the recommendations.
>
> It appears that these benchmarks are effectively crowdsourced. My
> primary concern is that there is no validation of the resulting
> recommendations so they run the risk of reinforcing current widespread
> bad practice as well as good practice.
>
> The only response CIS had was to join the CIS community and try and
> improve these guides. That is on my TODO list but it pains me to do so
> as they charge for commercial use. I don't like the idea of providing my
> expertise for free so someone else can make money out of it. I think I'd
> rather spend time on the security guides in the Tomcat docs.
>
> Hmm. Looking back at my notes it appears I looked into doing something
> similar back in 2010 and had similar concerns regarding the
> participation terms back then too.
>
> Regarding the suggestion to edit the contents of a JAR, that is a bad
> idea on so many levels.
>
> I'd argue that rather than spending time hiding the current Tomcat
> version - which is nothing more than security by obscurity - system
> admins should be investing time in an update process that allows easy
> updates (and roll-backs) to the Tomcat version. You only need to hide
> the version number if you aren't on top of your security updates. And if
> you aren't on top of your security updates you have bigger problems than
> needing to hide the version number.
>
> If you find you need to hide this version number to appease auditors -
> and using smarter auditors isn't an option - then there are a range of
> options as set out in the Tomcat 8.5 security guide. That guide also
> provides the correct way to override the version number (if you really
> need to) without editing the JAR contents. In short, you can simply
> override the individual file by placing at the right place in the file
> system:
>
> $CATALINA_BASE/lib/org/apache/catalina/util/ServerInfo.properties
>
> HTH,
>
> Mark
>
>
> On 05/09/2023 14:54, Robert Turner wrote:
> > Thanks Peter. Just to be clear that I have no concern about the goal of
> the
> > test -- only the method for obtaining the information, and the "implied"
> > correction.
> >
> > After going through the rest of the document I was provided, it seems to
> > "get more modern" as the questions go on -- suggesting the method of
> > improvement is additive, and possibly not corrective.
>
> Improvements are definitely corrective as well as additive. Early
> versions of the guide had very odd advice regarding MIME type mapping
> that has since been removed.
>
>
> >
> > On Tue, Sep 5, 2023 at 9:36 AM Peter Kreuser <lo...@kreuser.name> wrote:
> >
> >> Robert,
> >>
> >> While Mark Thomas will have a more detailled answer to this...
> >>
> >> The finding behind this test is valid (information disclosure with
> server
> >> version in responses), though the remediation listed here is from looong
> >> time ago, when the was no ErrorReportValve to purge the version info.
> >>
> >> So the CIS Tomcat 8(!) Guide is pretty outdated! Probably in more than
> >> this spot...
> >>
> >> Peter
> >>
> >>> Am 05.09.2023 um 14:03 schrieb Robert Turner <rt...@e-djuster.ca>:
> >>>
> >>> While I think I know the answer to my question, I wanted to
> double-check
> >>> with the group to confirm.
> >>>
> >>> I have been asked to perform the CIS Apache Tomcat 8 Benchmark (v1.1.0)
> >> on
> >>> our production Tomcat installation, and I am looking through the
> >> questions
> >>> / information extraction requests, and I suspect they are not really
> >>> evaluating what they think they are, and furthermore encouraging bad
> >>> practices.
> >>>
> >>> For instance, the first entry I have in the spreadsheet I was provided
> is
> >>> listed as follows:
> >>>
> >>> CIS Control:
> >>> 2.1 Alter the Advertised server.info String (Scored)
> >>>
> >>> Description:
> >>> The server.info attribute contains the name of the application
> service.
> >>> This value is presented to Tomcat clients when clients connect to the
> >>> tomcat server.
> >>>
> >>> Audit Procedures:
> >>> Perform the following to determine if the server.info value has been
> >>> changed:
> >>> Extract the ServerInfo.properties file and examine the server.info
> >>> attribute.
> >>> $ cd $CATALINA_HOME/lib
> >>> $ jar xf catalina.jar org/apache/catalina/util/ServerInfo.properties
> >>> $ grep server.info org/apache/catalina/util/ServerInfo.properties
> >>>
> >>>
> >>> So, other than a few issues with the audit procedures, etc. This seems
> to
> >>> be doing the following:
> >>>
> >>> a) evaluating a default value which I believe can be overridden and
> thus
> >>> may not actually reflect the value the server may provide to external
> >>> clients
> >>> b) encouraging the modification of the catalina.jar contents to correct
> >> the
> >>> default value
> >>>
> >>> There are a few similar items (for server.number, server.built) (2.2,
> >> 2.3).
> >>>
> >>>
> >>> Thoughts / comments from "those in the know"?
> >>>
> >>> Thanks,
> >>>
> >>> Robert
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> >> For additional commands, e-mail: users-help@tomcat.apache.org
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

Re: CIS Tomcat 8 Benchmark (v1.1.0) -- Questions

[Christopher Schultz <ch...@christopherschultz.net>]

Mark, Robert, et al,

On 9/5/23 11:26, Mark Thomas wrote:
> I spoke to some CIS representatives at a recent conference given that I 
> have concerns about the quality of some of the recommendations.
> 
> It appears that these benchmarks are effectively crowdsourced. My 
> primary concern is that there is no validation of the resulting 
> recommendations so they run the risk of reinforcing current widespread 
> bad practice as well as good practice.

This is interesting information. Lore really isn't a good source of 
security information. :( It makes me wonder why they are charging for 
that kind of information if it's basically the output of an internet 
search for "how do i secure a Tomcat installation".

> Regarding the suggestion to edit the contents of a JAR, that is a bad 
> idea on so many levels.

+1

Assuming that you are using signed JAR files, what you end up with is a 
JAR whose signature can no longer be validated. This would require that 
you disable JAR signing, which is probably another no-no when it comes 
to security -- and I'd agree -- if you have signed JARs you ought to be 
validating them. Another option would be to re-sign the modified JAR 
file with an internal certificate but that can present its own 
challenges. Most dev teams would just implement some kind of lazy 
auto-signing process with the cert and key sitting right on the server 
where it's being used, and at that point you are giving an attacker 
everything they need to work-around your security controls.

> I'd argue that rather than spending time hiding the current Tomcat 
> version - which is nothing more than security by obscurity - system 
> admins should be investing time in an update process that allows easy 
> updates (and roll-backs) to the Tomcat version. You only need to hide 
> the version number if you aren't on top of your security updates. And if 
> you aren't on top of your security updates you have bigger problems than 
> needing to hide the version number.

+1

https://tomcat.apache.org/presentations.html#latest-split-installation

> If you find you need to hide this version number to appease auditors - 
> and using smarter auditors isn't an option - then there are a range of 
> options as set out in the Tomcat 8.5 security guide. That guide also 
> provides the correct way to override the version number (if you really 
> need to) without editing the JAR contents. In short, you can simply 
> override the individual file by placing at the right place in the file 
> system:
> 
> $CATALINA_BASE/lib/org/apache/catalina/util/ServerInfo.properties

+1

The only argument /against/ doing that in the past that I've ever seen 
was that they "didn't want to pollute [their] Tomcat installation" which 
is crazy when the alternative is to alter a signed JAR provided by a 
trusted vendor.

This is also probably worth a read:
https://cwiki.apache.org/confluence/display/TOMCAT/Community+Review+of+DISA+STIG

I would bet that many of the items listed here have overlap with some of 
the more dubious items coming from CIS.

Hope that helps,
-chris


> On 05/09/2023 14:54, Robert Turner wrote:
>> Thanks Peter. Just to be clear that I have no concern about the goal 
>> of the
>> test -- only the method for obtaining the information, and the "implied"
>> correction.
>>
>> After going through the rest of the document I was provided, it seems to
>> "get more modern" as the questions go on -- suggesting the method of
>> improvement is additive, and possibly not corrective.
> 
> Improvements are definitely corrective as well as additive. Early 
> versions of the guide had very odd advice regarding MIME type mapping 
> that has since been removed.
> 
> 
>>
>> On Tue, Sep 5, 2023 at 9:36 AM Peter Kreuser <lo...@kreuser.name> wrote:
>>
>>> Robert,
>>>
>>> While Mark Thomas will have a more detailled answer to this...
>>>
>>> The finding behind this test is valid (information disclosure with 
>>> server
>>> version in responses), though the remediation listed here is from looong
>>> time ago, when the was no ErrorReportValve to purge the version info.
>>>
>>> So the CIS Tomcat 8(!) Guide is pretty outdated! Probably in more than
>>> this spot...
>>>
>>> Peter
>>>
>>>> Am 05.09.2023 um 14:03 schrieb Robert Turner <rt...@e-djuster.ca>:
>>>>
>>>> While I think I know the answer to my question, I wanted to 
>>>> double-check
>>>> with the group to confirm.
>>>>
>>>> I have been asked to perform the CIS Apache Tomcat 8 Benchmark (v1.1.0)
>>> on
>>>> our production Tomcat installation, and I am looking through the
>>> questions
>>>> / information extraction requests, and I suspect they are not really
>>>> evaluating what they think they are, and furthermore encouraging bad
>>>> practices.
>>>>
>>>> For instance, the first entry I have in the spreadsheet I was 
>>>> provided is
>>>> listed as follows:
>>>>
>>>> CIS Control:
>>>> 2.1 Alter the Advertised server.info String (Scored)
>>>>
>>>> Description:
>>>> The server.info attribute contains the name of the application service.
>>>> This value is presented to Tomcat clients when clients connect to the
>>>> tomcat server.
>>>>
>>>> Audit Procedures:
>>>> Perform the following to determine if the server.info value has been
>>>> changed:
>>>> Extract the ServerInfo.properties file and examine the server.info
>>>> attribute.
>>>> $ cd $CATALINA_HOME/lib
>>>> $ jar xf catalina.jar org/apache/catalina/util/ServerInfo.properties
>>>> $ grep server.info org/apache/catalina/util/ServerInfo.properties
>>>>
>>>>
>>>> So, other than a few issues with the audit procedures, etc. This 
>>>> seems to
>>>> be doing the following:
>>>>
>>>> a) evaluating a default value which I believe can be overridden and 
>>>> thus
>>>> may not actually reflect the value the server may provide to external
>>>> clients
>>>> b) encouraging the modification of the catalina.jar contents to correct
>>> the
>>>> default value
>>>>
>>>> There are a few similar items (for server.number, server.built) (2.2,
>>> 2.3).
>>>>
>>>>
>>>> Thoughts / comments from "those in the know"?
>>>>
>>>> Thanks,
>>>>
>>>> Robert
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: CIS Tomcat 8 Benchmark (v1.1.0) -- Questions

[Mark Thomas <ma...@apache.org>]

I spoke to some CIS representatives at a recent conference given that I 
have concerns about the quality of some of the recommendations.

It appears that these benchmarks are effectively crowdsourced. My 
primary concern is that there is no validation of the resulting 
recommendations so they run the risk of reinforcing current widespread 
bad practice as well as good practice.

The only response CIS had was to join the CIS community and try and 
improve these guides. That is on my TODO list but it pains me to do so 
as they charge for commercial use. I don't like the idea of providing my 
expertise for free so someone else can make money out of it. I think I'd 
rather spend time on the security guides in the Tomcat docs.

Hmm. Looking back at my notes it appears I looked into doing something 
similar back in 2010 and had similar concerns regarding the 
participation terms back then too.

Regarding the suggestion to edit the contents of a JAR, that is a bad 
idea on so many levels.

I'd argue that rather than spending time hiding the current Tomcat 
version - which is nothing more than security by obscurity - system 
admins should be investing time in an update process that allows easy 
updates (and roll-backs) to the Tomcat version. You only need to hide 
the version number if you aren't on top of your security updates. And if 
you aren't on top of your security updates you have bigger problems than 
needing to hide the version number.

If you find you need to hide this version number to appease auditors - 
and using smarter auditors isn't an option - then there are a range of 
options as set out in the Tomcat 8.5 security guide. That guide also 
provides the correct way to override the version number (if you really 
need to) without editing the JAR contents. In short, you can simply 
override the individual file by placing at the right place in the file 
system:

$CATALINA_BASE/lib/org/apache/catalina/util/ServerInfo.properties

HTH,

Mark


On 05/09/2023 14:54, Robert Turner wrote:
> Thanks Peter. Just to be clear that I have no concern about the goal of the
> test -- only the method for obtaining the information, and the "implied"
> correction.
> 
> After going through the rest of the document I was provided, it seems to
> "get more modern" as the questions go on -- suggesting the method of
> improvement is additive, and possibly not corrective.

Improvements are definitely corrective as well as additive. Early 
versions of the guide had very odd advice regarding MIME type mapping 
that has since been removed.


> 
> On Tue, Sep 5, 2023 at 9:36 AM Peter Kreuser <lo...@kreuser.name> wrote:
> 
>> Robert,
>>
>> While Mark Thomas will have a more detailled answer to this...
>>
>> The finding behind this test is valid (information disclosure with server
>> version in responses), though the remediation listed here is from looong
>> time ago, when the was no ErrorReportValve to purge the version info.
>>
>> So the CIS Tomcat 8(!) Guide is pretty outdated! Probably in more than
>> this spot...
>>
>> Peter
>>
>>> Am 05.09.2023 um 14:03 schrieb Robert Turner <rt...@e-djuster.ca>:
>>>
>>> While I think I know the answer to my question, I wanted to double-check
>>> with the group to confirm.
>>>
>>> I have been asked to perform the CIS Apache Tomcat 8 Benchmark (v1.1.0)
>> on
>>> our production Tomcat installation, and I am looking through the
>> questions
>>> / information extraction requests, and I suspect they are not really
>>> evaluating what they think they are, and furthermore encouraging bad
>>> practices.
>>>
>>> For instance, the first entry I have in the spreadsheet I was provided is
>>> listed as follows:
>>>
>>> CIS Control:
>>> 2.1 Alter the Advertised server.info String (Scored)
>>>
>>> Description:
>>> The server.info attribute contains the name of the application service.
>>> This value is presented to Tomcat clients when clients connect to the
>>> tomcat server.
>>>
>>> Audit Procedures:
>>> Perform the following to determine if the server.info value has been
>>> changed:
>>> Extract the ServerInfo.properties file and examine the server.info
>>> attribute.
>>> $ cd $CATALINA_HOME/lib
>>> $ jar xf catalina.jar org/apache/catalina/util/ServerInfo.properties
>>> $ grep server.info org/apache/catalina/util/ServerInfo.properties
>>>
>>>
>>> So, other than a few issues with the audit procedures, etc. This seems to
>>> be doing the following:
>>>
>>> a) evaluating a default value which I believe can be overridden and thus
>>> may not actually reflect the value the server may provide to external
>>> clients
>>> b) encouraging the modification of the catalina.jar contents to correct
>> the
>>> default value
>>>
>>> There are a few similar items (for server.number, server.built) (2.2,
>> 2.3).
>>>
>>>
>>> Thoughts / comments from "those in the know"?
>>>
>>> Thanks,
>>>
>>> Robert
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: CIS Tomcat 8 Benchmark (v1.1.0) -- Questions

[Robert Turner <rt...@e-djuster.ca>]

Thanks Peter. Just to be clear that I have no concern about the goal of the
test -- only the method for obtaining the information, and the "implied"
correction.

After going through the rest of the document I was provided, it seems to
"get more modern" as the questions go on -- suggesting the method of
improvement is additive, and possibly not corrective.

On Tue, Sep 5, 2023 at 9:36 AM Peter Kreuser <lo...@kreuser.name> wrote:

> Robert,
>
> While Mark Thomas will have a more detailled answer to this...
>
> The finding behind this test is valid (information disclosure with server
> version in responses), though the remediation listed here is from looong
> time ago, when the was no ErrorReportValve to purge the version info.
>
> So the CIS Tomcat 8(!) Guide is pretty outdated! Probably in more than
> this spot...
>
> Peter
>
> > Am 05.09.2023 um 14:03 schrieb Robert Turner <rt...@e-djuster.ca>:
> >
> > While I think I know the answer to my question, I wanted to double-check
> > with the group to confirm.
> >
> > I have been asked to perform the CIS Apache Tomcat 8 Benchmark (v1.1.0)
> on
> > our production Tomcat installation, and I am looking through the
> questions
> > / information extraction requests, and I suspect they are not really
> > evaluating what they think they are, and furthermore encouraging bad
> > practices.
> >
> > For instance, the first entry I have in the spreadsheet I was provided is
> > listed as follows:
> >
> > CIS Control:
> > 2.1 Alter the Advertised server.info String (Scored)
> >
> > Description:
> > The server.info attribute contains the name of the application service.
> > This value is presented to Tomcat clients when clients connect to the
> > tomcat server.
> >
> > Audit Procedures:
> > Perform the following to determine if the server.info value has been
> > changed:
> > Extract the ServerInfo.properties file and examine the server.info
> > attribute.
> > $ cd $CATALINA_HOME/lib
> > $ jar xf catalina.jar org/apache/catalina/util/ServerInfo.properties
> > $ grep server.info org/apache/catalina/util/ServerInfo.properties
> >
> >
> > So, other than a few issues with the audit procedures, etc. This seems to
> > be doing the following:
> >
> > a) evaluating a default value which I believe can be overridden and thus
> > may not actually reflect the value the server may provide to external
> > clients
> > b) encouraging the modification of the catalina.jar contents to correct
> the
> > default value
> >
> > There are a few similar items (for server.number, server.built) (2.2,
> 2.3).
> >
> >
> > Thoughts / comments from "those in the know"?
> >
> > Thanks,
> >
> > Robert
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

Re: CIS Tomcat 8 Benchmark (v1.1.0) -- Questions

[Peter Kreuser <lo...@kreuser.name>]

Robert,

While Mark Thomas will have a more detailled answer to this...

The finding behind this test is valid (information disclosure with server version in responses), though the remediation listed here is from looong time ago, when the was no ErrorReportValve to purge the version info.

So the CIS Tomcat 8(!) Guide is pretty outdated! Probably in more than this spot...

Peter

> Am 05.09.2023 um 14:03 schrieb Robert Turner <rt...@e-djuster.ca>:
> 
> While I think I know the answer to my question, I wanted to double-check
> with the group to confirm.
> 
> I have been asked to perform the CIS Apache Tomcat 8 Benchmark (v1.1.0) on
> our production Tomcat installation, and I am looking through the questions
> / information extraction requests, and I suspect they are not really
> evaluating what they think they are, and furthermore encouraging bad
> practices.
> 
> For instance, the first entry I have in the spreadsheet I was provided is
> listed as follows:
> 
> CIS Control:
> 2.1 Alter the Advertised server.info String (Scored)
> 
> Description:
> The server.info attribute contains the name of the application service.
> This value is presented to Tomcat clients when clients connect to the
> tomcat server.
> 
> Audit Procedures:
> Perform the following to determine if the server.info value has been
> changed:
> Extract the ServerInfo.properties file and examine the server.info
> attribute.
> $ cd $CATALINA_HOME/lib
> $ jar xf catalina.jar org/apache/catalina/util/ServerInfo.properties
> $ grep server.info org/apache/catalina/util/ServerInfo.properties
> 
> 
> So, other than a few issues with the audit procedures, etc. This seems to
> be doing the following:
> 
> a) evaluating a default value which I believe can be overridden and thus
> may not actually reflect the value the server may provide to external
> clients
> b) encouraging the modification of the catalina.jar contents to correct the
> default value
> 
> There are a few similar items (for server.number, server.built) (2.2, 2.3).
> 
> 
> Thoughts / comments from "those in the know"?
> 
> Thanks,
> 
> Robert

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org