EngineLifecycleTest and ReuseAddress and mysterious systest failures

[Benson Margulies <bi...@basistech.com>]

Let me try to tie a few threads together here.

1) Originally, I was having problems with 'servers that wouldn't die' in
my own application code. 

1a) A bug with timer threads in jetty. Perhaps provoked by some CXF
quirk, but not our job.
1b) the need to explicitly shutdown the bus in addition to calling close
and/or destroy on the containing application context. This might deserve
its own JIRA.

2) In an effort to shed light on #1, I added a more complex test case to
EngineLifecycleTest that uses a more complex Jetty context and adds some
servlets. It gets some bind exceptions that might indicate something
interesting, or might just be more changes rung on the bell of
SO_REUSEADDR.

3) Any reliable attempt to test code that binds ports needs to either
use many different ports or use SO_REUSEADDR. We have a design decision
here.

3a) Make CXF always set reuse address if it can. Essentially, that would
call for code that recognized the well-known Jetty connector classes,
and, if one was in use, made the call to turn it on.

3b) Same as 3a, but add some attribute to the engine extensibility XML
to disable this behavior, for those rare birds who care about the
security issue.

3c) Run all our own tests with an explicit configuration that gets
around this, such as a custom connection class that always turned it on.

Once we do \something/ about this, we can turn off Dan's crowbar.

4) The murky question of how we manage to stop the server, start the
server, and end up with a bio connector that hasn't created its socket
yet. I wallpapered this with a call to .open() in the test, but I
strongly suggest that this reflects something more interesting. What it
might reflect, however, is the failure to use REUSEADDR in all the
\other/ tests. So perhaps the strategy should be to address (ahem) #3
and then see if we can still produce something like this.

--benson

Re: EngineLifecycleTest and ReuseAddress and mysterious systest failures

[Willem Jiang <ni...@iona.com>]

Hi  Benson,


I will work on the issue3, adding a attribute in the JettyEngine's 
configuration to set the SO_REUSEADDR flage.
Then We can turn off Dan's crowbar.
Here is the JIRA[1] which traces it.

[1] https://issues.apache.org/jira/browse/CXF-1134

Willem.

Benson Margulies wrote:
> Let me try to tie a few threads together here.
>
> 1) Originally, I was having problems with 'servers that wouldn't die' in
> my own application code. 
>
> 1a) A bug with timer threads in jetty. Perhaps provoked by some CXF
> quirk, but not our job.
> 1b) the need to explicitly shutdown the bus in addition to calling close
> and/or destroy on the containing application context. This might deserve
> its own JIRA.
>
> 2) In an effort to shed light on #1, I added a more complex test case to
> EngineLifecycleTest that uses a more complex Jetty context and adds some
> servlets. It gets some bind exceptions that might indicate something
> interesting, or might just be more changes rung on the bell of
> SO_REUSEADDR.
>
> 3) Any reliable attempt to test code that binds ports needs to either
> use many different ports or use SO_REUSEADDR. We have a design decision
> here.
>
> 3a) Make CXF always set reuse address if it can. Essentially, that would
> call for code that recognized the well-known Jetty connector classes,
> and, if one was in use, made the call to turn it on.
>
> 3b) Same as 3a, but add some attribute to the engine extensibility XML
> to disable this behavior, for those rare birds who care about the
> security issue.
>
> 3c) Run all our own tests with an explicit configuration that gets
> around this, such as a custom connection class that always turned it on.
>
> Once we do \something/ about this, we can turn off Dan's crowbar.
>
> 4) The murky question of how we manage to stop the server, start the
> server, and end up with a bio connector that hasn't created its socket
> yet. I wallpapered this with a call to .open() in the test, but I
> strongly suggest that this reflects something more interesting. What it
> might reflect, however, is the failure to use REUSEADDR in all the
> \other/ tests. So perhaps the strategy should be to address (ahem) #3
> and then see if we can still produce something like this.
>
> --benson
>
>