You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Jesse Schulman <je...@dreamtsoft.com> on 2017/01/05 21:05:32 UTC
tomcat-embed 8.5.9 - runtime changes to SSLHostConfig objects
We are using tomcat-embed 8.5.9, java8 and running on Centos7. Given
Tomcat's new support for SNI, we wish to support adding/removing/updating
certificates via our application at runtime without restarting tomcat or
binding/unbinding the port.
Our configuration is very simple, we have a single servlet for all
requests, so we have a single connector/endpoint to manage all
SSLHostConfigs.
It appears that by manipulating the list of SSLHostConfig objects in the
AbstractEndpoint we can achieve what we want, there however don't appear to
be any public methods available that allow that kind of operation.
I was able to extend a few tomcat classes (Connector,
AbstractHttp11JsseProtocol, NioEndpoint) to expose what I need and verify
that I can change the SSLHostConfig at runtime, however I would prefer to
use APIs fully supported by tomcat.
Is there any way to do what I want with the currently available APIs, or
are there any plans to expose this kind of functionality?
If not, are there any risks or issues with taking the approach described
above by extending classes to expose what I need?
Thanks!
Jesse
Re: tomcat-embed 8.5.9 - runtime changes to SSLHostConfig objects
Posted by Jesse Schulman <je...@dreamtsoft.com>.
On Mon, Jan 9, 2017 at 1:53 PM Mark Thomas <ma...@apache.org> wrote:
> On 09/01/2017 20:13, Jesse Schulman wrote:
> > On Thu, Jan 5, 2017 at 9:48 PM Jesse Schulman <je...@dreamtsoft.com>
> wrote:
> >
> >> On Thu, Jan 5, 2017 at 2:08 PM Mark Thomas <ma...@apache.org> wrote:
> >>
> >> On 05/01/2017 21:05, Jesse Schulman wrote:
> >>> We are using tomcat-embed 8.5.9, java8 and running on Centos7. Given
> >>> Tomcat's new support for SNI, we wish to support
> adding/removing/updating
> >>> certificates via our application at runtime without restarting tomcat
> or
> >>> binding/unbinding the port.
> >>>
> >>> Our configuration is very simple, we have a single servlet for all
> >>> requests, so we have a single connector/endpoint to manage all
> >>> SSLHostConfigs.
> >>>
> >>> It appears that by manipulating the list of SSLHostConfig objects in
> the
> >>> AbstractEndpoint we can achieve what we want, there however don't
> appear
> >> to
> >>> be any public methods available that allow that kind of operation.
> >>
> >> It should be possible with the current API. What can't you do?
> >>
> >>
> >> I don't think I can modify an existing/added SSLHostConfig (remove an
> old
> >> expiring certificate and add a new certificate). I also don't think I
> can
> >> remove the SSLHostConfig for a given SNI hostname once it has been
> added.
> >>
> >>> I was able to extend a few tomcat classes (Connector,
> >>> AbstractHttp11JsseProtocol, NioEndpoint) to expose what I need and
> verify
> >>> that I can change the SSLHostConfig at runtime, however I would prefer
> to
> >>> use APIs fully supported by tomcat.
> >>>
> >>> Is there any way to do what I want with the currently available APIs,
> or
> >>> are there any plans to expose this kind of functionality?
> >>
> >> It depends exactly what you want to do.
> >>
> >> AbstractEndpoint.addSslHostConfig()
> >> AbstractEndpoint.findSslHostConfigs()
> >>
> >> should be enough.
> >>
> >>
> >> It seems addSslHostConfig does a putIfAbsent, which means I can't
> replace
> >> an existing SSLHostConfig to update the certificate for an SNI hostname
> (or
> >> the default SSLHostConfig). I also can't remove an SSLHostConfig
> entirely,
> >> which is something we'd like to support in our application. For me it
> >> would be simplest to create a new SSLHostConfig and replace the old one
> >> when there is a need to update a certificate.
> >>
> >>
> >>
> >>> If not, are there any risks or issues with taking the approach
> described
> >>> above by extending classes to expose what I need?
> >>
> >> It depends what you want to do. Generally, there is a risk we'll change
> >> an API you are depending on since a lot of those are treated as internal
> >> APIs. Some sample code might help.
> >>
> >>
> >> I would of course test when we upgrade tomcat, and leverage any new APIs
> >> that would allow me to remove any of my custom code.
> >>
> >> I extended AbstractHttp11JsseProtocol just so I could control which
> >> endpoint implementation was used and so I could get access to my
> endpoint
> >> implementation. My protocol is basically a copy of Http11NioProtocol
> that
> >> constructs my endpoint implementation and has a getter for the same.
> >>
> >> The endpoint is where I added functionality and looks like this:
> >>
> >> public class MyNioEndpoint extends NioEndpoint {
> >> private static final MyLogger LOGGER =
> >> MyLogger.getLogger(MyNioEndpoint.class.getName());
> >>
> >> public void removeSSLHostConfig(String sniHostName) {
> >> if (Strings.isNullOrEmpty(sniHostName)) {
> >> LOGGER.error("Cannot remove host config for invalid
> hostname:
> >> " + sniHostName);
> >> return;
> >> }
> >>
> >> if (sniHostName.equals(getDefaultSSLHostConfigName())) {
> >> LOGGER.error("Cannot remove default SSLHostConfig");
> >> return;
> >> }
> >>
> >> SSLHostConfig removed = sslHostConfigs.remove(sniHostName);
> >> if (removed != null)
> >> releaseSSLContext(removed);
> >> }
> >>
> >> public void addOrUpdateSSLHostConfig(SSLHostConfig config) {
> >> if (config == null) {
> >> LOGGER.error("null SSLHostConfig provided");
> >> return;
> >> }
> >>
> >> String hostName = config.getHostName();
> >> if (Strings.isNullOrEmpty(hostName)) {
> >> LOGGER.error("Invalid SSLHostConfig provided, cannot
> >> add/update, hostname was empty");
> >> return;
> >> }
> >>
> >> for (SSLHostConfig loadedConfig : findSslHostConfigs()) {
> >> if (hostName.equals(loadedConfig.getHostName())) {
> >> sslHostConfigs.remove(hostName);
> >> releaseSSLContext(loadedConfig);
> >> addSslHostConfig(loadedConfig);
> >> return;
> >> }
> >> }
> >>
> >> addSslHostConfig(config);
> >> }
> >>
> >> }
> >>
> >> Thanks for the reply, your help/suggestions are much appreciated!
> >> Jesse
> >>
> >>
> > I am bumping this, I think I made it sound as if I had all my questions
> > answered (since I didn't put any questions in my response). Here are my
> > remaining questions:
> >
> > Is there an existing/supported way to remove/replace an existing
> > SSLHostConfig in AbstractEndpoint that is exposed via the Connector
> similar
> > to how addSslHostConfig and findSslHostConfigs are exposed? Such that I
> > don't need the MyEndpoint class I shared above?
>
> No.
>
> > If the answer to the first question is no, are there any concerns with
> the
> > code I shared? By releasing the SSLContext is that ensuring that nothing
> > will stick around from an SSLHostConfig that is removed/replaced, or is
> > there more to do?
>
> Yes, when using APR. You can't release the SSLContext until every
> connection using the SSLContext is closed else you are likely to get a
> JVM crash.
>
> I recommend opening an enhancement request to make removal of an
> SSLContext possible. Patches welcome ;)
>
> Thank you for the response.
Since we're using NIO and only extending the NIOEndpoint to add our
functionality, I'll consider what we're doing to be safe.
I will open an enhancement request as you suggest and track it's
progress. I'll of course do testing for any future tomcat upgrade we do,
hopefully our next upgrade can be to a version which includes the
enhancement :)
I can consider helping with a patch if that makes sense but I am not sure
how it would be best to handle the APR endpoint case. I can imagine moving
the SSLHostConfig objects to an inactive collection and then
watching/waiting for their SSLContext to have all connections closed but
I'm not sure where best to check for closed connections, a separate thread
or is there an internal thread/event that would be appropriate to perform
the checking on some interval? Websocket connections could technically
live indefinitely, which may suggest a configurable timeout should exist
after which those connections would be forcibly closed?
Thanks again!
Jesse
>
> Mark
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: tomcat-embed 8.5.9 - runtime changes to SSLHostConfig objects
Posted by Mark Thomas <ma...@apache.org>.
On 09/01/2017 20:13, Jesse Schulman wrote:
> On Thu, Jan 5, 2017 at 9:48 PM Jesse Schulman <je...@dreamtsoft.com> wrote:
>
>> On Thu, Jan 5, 2017 at 2:08 PM Mark Thomas <ma...@apache.org> wrote:
>>
>> On 05/01/2017 21:05, Jesse Schulman wrote:
>>> We are using tomcat-embed 8.5.9, java8 and running on Centos7. Given
>>> Tomcat's new support for SNI, we wish to support adding/removing/updating
>>> certificates via our application at runtime without restarting tomcat or
>>> binding/unbinding the port.
>>>
>>> Our configuration is very simple, we have a single servlet for all
>>> requests, so we have a single connector/endpoint to manage all
>>> SSLHostConfigs.
>>>
>>> It appears that by manipulating the list of SSLHostConfig objects in the
>>> AbstractEndpoint we can achieve what we want, there however don't appear
>> to
>>> be any public methods available that allow that kind of operation.
>>
>> It should be possible with the current API. What can't you do?
>>
>>
>> I don't think I can modify an existing/added SSLHostConfig (remove an old
>> expiring certificate and add a new certificate). I also don't think I can
>> remove the SSLHostConfig for a given SNI hostname once it has been added.
>>
>>> I was able to extend a few tomcat classes (Connector,
>>> AbstractHttp11JsseProtocol, NioEndpoint) to expose what I need and verify
>>> that I can change the SSLHostConfig at runtime, however I would prefer to
>>> use APIs fully supported by tomcat.
>>>
>>> Is there any way to do what I want with the currently available APIs, or
>>> are there any plans to expose this kind of functionality?
>>
>> It depends exactly what you want to do.
>>
>> AbstractEndpoint.addSslHostConfig()
>> AbstractEndpoint.findSslHostConfigs()
>>
>> should be enough.
>>
>>
>> It seems addSslHostConfig does a putIfAbsent, which means I can't replace
>> an existing SSLHostConfig to update the certificate for an SNI hostname (or
>> the default SSLHostConfig). I also can't remove an SSLHostConfig entirely,
>> which is something we'd like to support in our application. For me it
>> would be simplest to create a new SSLHostConfig and replace the old one
>> when there is a need to update a certificate.
>>
>>
>>
>>> If not, are there any risks or issues with taking the approach described
>>> above by extending classes to expose what I need?
>>
>> It depends what you want to do. Generally, there is a risk we'll change
>> an API you are depending on since a lot of those are treated as internal
>> APIs. Some sample code might help.
>>
>>
>> I would of course test when we upgrade tomcat, and leverage any new APIs
>> that would allow me to remove any of my custom code.
>>
>> I extended AbstractHttp11JsseProtocol just so I could control which
>> endpoint implementation was used and so I could get access to my endpoint
>> implementation. My protocol is basically a copy of Http11NioProtocol that
>> constructs my endpoint implementation and has a getter for the same.
>>
>> The endpoint is where I added functionality and looks like this:
>>
>> public class MyNioEndpoint extends NioEndpoint {
>> private static final MyLogger LOGGER =
>> MyLogger.getLogger(MyNioEndpoint.class.getName());
>>
>> public void removeSSLHostConfig(String sniHostName) {
>> if (Strings.isNullOrEmpty(sniHostName)) {
>> LOGGER.error("Cannot remove host config for invalid hostname:
>> " + sniHostName);
>> return;
>> }
>>
>> if (sniHostName.equals(getDefaultSSLHostConfigName())) {
>> LOGGER.error("Cannot remove default SSLHostConfig");
>> return;
>> }
>>
>> SSLHostConfig removed = sslHostConfigs.remove(sniHostName);
>> if (removed != null)
>> releaseSSLContext(removed);
>> }
>>
>> public void addOrUpdateSSLHostConfig(SSLHostConfig config) {
>> if (config == null) {
>> LOGGER.error("null SSLHostConfig provided");
>> return;
>> }
>>
>> String hostName = config.getHostName();
>> if (Strings.isNullOrEmpty(hostName)) {
>> LOGGER.error("Invalid SSLHostConfig provided, cannot
>> add/update, hostname was empty");
>> return;
>> }
>>
>> for (SSLHostConfig loadedConfig : findSslHostConfigs()) {
>> if (hostName.equals(loadedConfig.getHostName())) {
>> sslHostConfigs.remove(hostName);
>> releaseSSLContext(loadedConfig);
>> addSslHostConfig(loadedConfig);
>> return;
>> }
>> }
>>
>> addSslHostConfig(config);
>> }
>>
>> }
>>
>> Thanks for the reply, your help/suggestions are much appreciated!
>> Jesse
>>
>>
> I am bumping this, I think I made it sound as if I had all my questions
> answered (since I didn't put any questions in my response). Here are my
> remaining questions:
>
> Is there an existing/supported way to remove/replace an existing
> SSLHostConfig in AbstractEndpoint that is exposed via the Connector similar
> to how addSslHostConfig and findSslHostConfigs are exposed? Such that I
> don't need the MyEndpoint class I shared above?
No.
> If the answer to the first question is no, are there any concerns with the
> code I shared? By releasing the SSLContext is that ensuring that nothing
> will stick around from an SSLHostConfig that is removed/replaced, or is
> there more to do?
Yes, when using APR. You can't release the SSLContext until every
connection using the SSLContext is closed else you are likely to get a
JVM crash.
I recommend opening an enhancement request to make removal of an
SSLContext possible. Patches welcome ;)
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: tomcat-embed 8.5.9 - runtime changes to SSLHostConfig objects
Posted by Jesse Schulman <je...@dreamtsoft.com>.
On Thu, Jan 5, 2017 at 9:48 PM Jesse Schulman <je...@dreamtsoft.com> wrote:
> On Thu, Jan 5, 2017 at 2:08 PM Mark Thomas <ma...@apache.org> wrote:
>
> On 05/01/2017 21:05, Jesse Schulman wrote:
> > We are using tomcat-embed 8.5.9, java8 and running on Centos7. Given
> > Tomcat's new support for SNI, we wish to support adding/removing/updating
> > certificates via our application at runtime without restarting tomcat or
> > binding/unbinding the port.
> >
> > Our configuration is very simple, we have a single servlet for all
> > requests, so we have a single connector/endpoint to manage all
> > SSLHostConfigs.
> >
> > It appears that by manipulating the list of SSLHostConfig objects in the
> > AbstractEndpoint we can achieve what we want, there however don't appear
> to
> > be any public methods available that allow that kind of operation.
>
> It should be possible with the current API. What can't you do?
>
>
> I don't think I can modify an existing/added SSLHostConfig (remove an old
> expiring certificate and add a new certificate). I also don't think I can
> remove the SSLHostConfig for a given SNI hostname once it has been added.
>
> > I was able to extend a few tomcat classes (Connector,
> > AbstractHttp11JsseProtocol, NioEndpoint) to expose what I need and verify
> > that I can change the SSLHostConfig at runtime, however I would prefer to
> > use APIs fully supported by tomcat.
> >
> > Is there any way to do what I want with the currently available APIs, or
> > are there any plans to expose this kind of functionality?
>
> It depends exactly what you want to do.
>
> AbstractEndpoint.addSslHostConfig()
> AbstractEndpoint.findSslHostConfigs()
>
> should be enough.
>
>
> It seems addSslHostConfig does a putIfAbsent, which means I can't replace
> an existing SSLHostConfig to update the certificate for an SNI hostname (or
> the default SSLHostConfig). I also can't remove an SSLHostConfig entirely,
> which is something we'd like to support in our application. For me it
> would be simplest to create a new SSLHostConfig and replace the old one
> when there is a need to update a certificate.
>
>
>
> > If not, are there any risks or issues with taking the approach described
> > above by extending classes to expose what I need?
>
> It depends what you want to do. Generally, there is a risk we'll change
> an API you are depending on since a lot of those are treated as internal
> APIs. Some sample code might help.
>
>
> I would of course test when we upgrade tomcat, and leverage any new APIs
> that would allow me to remove any of my custom code.
>
> I extended AbstractHttp11JsseProtocol just so I could control which
> endpoint implementation was used and so I could get access to my endpoint
> implementation. My protocol is basically a copy of Http11NioProtocol that
> constructs my endpoint implementation and has a getter for the same.
>
> The endpoint is where I added functionality and looks like this:
>
> public class MyNioEndpoint extends NioEndpoint {
> private static final MyLogger LOGGER =
> MyLogger.getLogger(MyNioEndpoint.class.getName());
>
> public void removeSSLHostConfig(String sniHostName) {
> if (Strings.isNullOrEmpty(sniHostName)) {
> LOGGER.error("Cannot remove host config for invalid hostname:
> " + sniHostName);
> return;
> }
>
> if (sniHostName.equals(getDefaultSSLHostConfigName())) {
> LOGGER.error("Cannot remove default SSLHostConfig");
> return;
> }
>
> SSLHostConfig removed = sslHostConfigs.remove(sniHostName);
> if (removed != null)
> releaseSSLContext(removed);
> }
>
> public void addOrUpdateSSLHostConfig(SSLHostConfig config) {
> if (config == null) {
> LOGGER.error("null SSLHostConfig provided");
> return;
> }
>
> String hostName = config.getHostName();
> if (Strings.isNullOrEmpty(hostName)) {
> LOGGER.error("Invalid SSLHostConfig provided, cannot
> add/update, hostname was empty");
> return;
> }
>
> for (SSLHostConfig loadedConfig : findSslHostConfigs()) {
> if (hostName.equals(loadedConfig.getHostName())) {
> sslHostConfigs.remove(hostName);
> releaseSSLContext(loadedConfig);
> addSslHostConfig(loadedConfig);
> return;
> }
> }
>
> addSslHostConfig(config);
> }
>
> }
>
> Thanks for the reply, your help/suggestions are much appreciated!
> Jesse
>
>
I am bumping this, I think I made it sound as if I had all my questions
answered (since I didn't put any questions in my response). Here are my
remaining questions:
Is there an existing/supported way to remove/replace an existing
SSLHostConfig in AbstractEndpoint that is exposed via the Connector similar
to how addSslHostConfig and findSslHostConfigs are exposed? Such that I
don't need the MyEndpoint class I shared above?
If the answer to the first question is no, are there any concerns with the
code I shared? By releasing the SSLContext is that ensuring that nothing
will stick around from an SSLHostConfig that is removed/replaced, or is
there more to do?
Thanks!
Jesse
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: tomcat-embed 8.5.9 - runtime changes to SSLHostConfig objects
Posted by Jesse Schulman <je...@dreamtsoft.com>.
On Thu, Jan 5, 2017 at 2:08 PM Mark Thomas <ma...@apache.org> wrote:
> On 05/01/2017 21:05, Jesse Schulman wrote:
> > We are using tomcat-embed 8.5.9, java8 and running on Centos7. Given
> > Tomcat's new support for SNI, we wish to support adding/removing/updating
> > certificates via our application at runtime without restarting tomcat or
> > binding/unbinding the port.
> >
> > Our configuration is very simple, we have a single servlet for all
> > requests, so we have a single connector/endpoint to manage all
> > SSLHostConfigs.
> >
> > It appears that by manipulating the list of SSLHostConfig objects in the
> > AbstractEndpoint we can achieve what we want, there however don't appear
> to
> > be any public methods available that allow that kind of operation.
>
> It should be possible with the current API. What can't you do?
>
>
I don't think I can modify an existing/added SSLHostConfig (remove an old
expiring certificate and add a new certificate). I also don't think I can
remove the SSLHostConfig for a given SNI hostname once it has been added.
> I was able to extend a few tomcat classes (Connector,
> > AbstractHttp11JsseProtocol, NioEndpoint) to expose what I need and verify
> > that I can change the SSLHostConfig at runtime, however I would prefer to
> > use APIs fully supported by tomcat.
> >
> > Is there any way to do what I want with the currently available APIs, or
> > are there any plans to expose this kind of functionality?
>
> It depends exactly what you want to do.
>
> AbstractEndpoint.addSslHostConfig()
> AbstractEndpoint.findSslHostConfigs()
>
> should be enough.
>
It seems addSslHostConfig does a putIfAbsent, which means I can't replace
an existing SSLHostConfig to update the certificate for an SNI hostname (or
the default SSLHostConfig). I also can't remove an SSLHostConfig entirely,
which is something we'd like to support in our application. For me it
would be simplest to create a new SSLHostConfig and replace the old one
when there is a need to update a certificate.
>
> > If not, are there any risks or issues with taking the approach described
> > above by extending classes to expose what I need?
>
> It depends what you want to do. Generally, there is a risk we'll change
> an API you are depending on since a lot of those are treated as internal
> APIs. Some sample code might help.
>
I would of course test when we upgrade tomcat, and leverage any new APIs
that would allow me to remove any of my custom code.
I extended AbstractHttp11JsseProtocol just so I could control which
endpoint implementation was used and so I could get access to my endpoint
implementation. My protocol is basically a copy of Http11NioProtocol that
constructs my endpoint implementation and has a getter for the same.
The endpoint is where I added functionality and looks like this:
public class MyNioEndpoint extends NioEndpoint {
private static final MyLogger LOGGER =
MyLogger.getLogger(MyNioEndpoint.class.getName());
public void removeSSLHostConfig(String sniHostName) {
if (Strings.isNullOrEmpty(sniHostName)) {
LOGGER.error("Cannot remove host config for invalid hostname: "
+ sniHostName);
return;
}
if (sniHostName.equals(getDefaultSSLHostConfigName())) {
LOGGER.error("Cannot remove default SSLHostConfig");
return;
}
SSLHostConfig removed = sslHostConfigs.remove(sniHostName);
if (removed != null)
releaseSSLContext(removed);
}
public void addOrUpdateSSLHostConfig(SSLHostConfig config) {
if (config == null) {
LOGGER.error("null SSLHostConfig provided");
return;
}
String hostName = config.getHostName();
if (Strings.isNullOrEmpty(hostName)) {
LOGGER.error("Invalid SSLHostConfig provided, cannot
add/update, hostname was empty");
return;
}
for (SSLHostConfig loadedConfig : findSslHostConfigs()) {
if (hostName.equals(loadedConfig.getHostName())) {
sslHostConfigs.remove(hostName);
releaseSSLContext(loadedConfig);
addSslHostConfig(loadedConfig);
return;
}
}
addSslHostConfig(config);
}
}
Thanks for the reply, your help/suggestions are much appreciated!
Jesse
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: tomcat-embed 8.5.9 - runtime changes to SSLHostConfig objects
Posted by Mark Thomas <ma...@apache.org>.
On 05/01/2017 21:05, Jesse Schulman wrote:
> We are using tomcat-embed 8.5.9, java8 and running on Centos7. Given
> Tomcat's new support for SNI, we wish to support adding/removing/updating
> certificates via our application at runtime without restarting tomcat or
> binding/unbinding the port.
>
> Our configuration is very simple, we have a single servlet for all
> requests, so we have a single connector/endpoint to manage all
> SSLHostConfigs.
>
> It appears that by manipulating the list of SSLHostConfig objects in the
> AbstractEndpoint we can achieve what we want, there however don't appear to
> be any public methods available that allow that kind of operation.
It should be possible with the current API. What can't you do?
> I was able to extend a few tomcat classes (Connector,
> AbstractHttp11JsseProtocol, NioEndpoint) to expose what I need and verify
> that I can change the SSLHostConfig at runtime, however I would prefer to
> use APIs fully supported by tomcat.
>
> Is there any way to do what I want with the currently available APIs, or
> are there any plans to expose this kind of functionality?
It depends exactly what you want to do.
AbstractEndpoint.addSslHostConfig()
AbstractEndpoint.findSslHostConfigs()
should be enough.
> If not, are there any risks or issues with taking the approach described
> above by extending classes to expose what I need?
It depends what you want to do. Generally, there is a risk we'll change
an API you are depending on since a lot of those are treated as internal
APIs. Some sample code might help.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org