You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cxf.apache.org by "Benson Margulies (JIRA)" <ji...@apache.org> on 2007/11/19 04:16:43 UTC
[jira] Created: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Address in use errors from attempt to repeatedly start and stop services
------------------------------------------------------------------------
Key: CXF-1217
URL: https://issues.apache.org/jira/browse/CXF-1217
Project: CXF
Issue Type: Bug
Components: Transports
Affects Versions: 2.1
Environment: Any
Reporter: Benson Margulies
Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
WARNING: failed Server@1174031
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind(Native Method)
which eventually, as far as I can tell, turns into:
INFO: doOpen POST http://localhost:8808/Greeter false
Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
SEVERE: IO error reading response
java.io.FileNotFoundException: http://localhost:8808/Greeter
On the other hand, all is well in Eclipse.
I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544761 ]
Gary Tully commented on CXF-1217:
---------------------------------
Hi Willlem,
with 6.1.6 (rc1) SO_REUSEADDR defaults to true. I guess in JettyHTTPServerEngine.java we need to apply our version of the reuseAddr flag before we call server.start(). The puropose of the flag would be to allow a user to turn SO_REUSEADDR off (as it defaults to true).
I will attach a small patch that will allow a user to turn SO_REUSEADDR off with 6.1.6. The 6.1.6 should be in a maven repo shortly, it was released a few days ago. this patch will require 6.1.6
There is still a problem with shutdown however, and it does appear to be related to keepAlive and the client sockets. I am still tracking it down.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Assignee: willem Jiang
> Attachments: busShutdownTest.patch.txt, jetty6.1.6.patch
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
willem Jiang resolved CXF-1217.
-------------------------------
Resolution: Fixed
Fix Version/s: 2.1
2.0.4
patches are in the trunk and 2.0.x-fixes branch.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Assignee: willem Jiang
> Fix For: 2.0.4, 2.1
>
> Attachments: busShutdownTest.patch.txt, jetty6.1.6.patch
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CXF-1217) Address in use errors from
attempt to repeatedly start and stop services
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544498 ]
gtully edited comment on CXF-1217 at 11/21/07 7:25 AM:
-----------------------------------------------------------
This "open" call has been fixed in 6.1.6: http://svn.codehaus.org/jetty/jetty/tags/jetty-6.1.6/modules/jetty/src/main/java/org/mortbay/jetty/nio/SelectChannelConnector.java so we may be in luck with an upgrade once the option is set by CXF in the right place.
was (Author: gtully):
This "open" call has been fixed in 2.1.6: http://svn.codehaus.org/jetty/jetty/tags/jetty-6.1.6/modules/jetty/src/main/java/org/mortbay/jetty/nio/SelectChannelConnector.java so we may be in luck with an upgrade once the option is set by CXF in the right place.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Attachments: busShutdownTest.patch.txt
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Tully updated CXF-1217:
----------------------------
Attachment: busShutdownTest.patch.txt
This patch contains a broken system test that uses a bus, shutsdown the bus and attempts to reuse the bus with the same server http url. The client in the test hangs on the second invocation.
I think this test represents a valid expection that should work out of the box for CXF. bus.shutdown should leave the JVM in the state that it was before bus initialisation, pending gc.
The test fails repeatidly for me on XP both in eclipse and with mvn.
Both using a different server side port on the second run and using the system property org.apache.cxf.transports.http_jetty.DontClosePort work to make the test pass but IMHO both should not be necessary.
Working through this test case may help to provide the real culprit that is responsible for the introduction of org.apache.cxf.transports.http_jetty.DontClosePort.
Ps: Benson, I don't mean to hijack this bug report, I just want to focus the issue. I think the system property workaround is really overkill and problematic from a test perspective as it is a system global.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Attachments: busShutdownTest.patch.txt
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "Benson Margulies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544494 ]
Benson Margulies commented on CXF-1217:
---------------------------------------
Maybe we really need to look at the new API in the Jetty release candidate that purports (according to their response to my JIRA) to provide an orderly process for this purpose?
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Attachments: busShutdownTest.patch.txt
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544859 ]
Gary Tully commented on CXF-1217:
---------------------------------
So I finally got to the root of this problem. It is a combination of keep-alive and chunking and stream wrapping.
Keepalive is on by default and turned on in HTTPClient by default for http 1.1 responses. The keepalive check interval is 5 seconds so it there is a sleep in the code, we don't end up reusing a keepalive connection and everything works. In the BusShutdownTest there is no point to keepalive because the server goes away after the first request so the socket is always in close_wait when the client attempts to use it.
When the client gets a cached/keep-alive socket, the write succeeds and the read response fails because the server side of the socket has been closed. The retry logic in HttpClient attempts to close and reopen the connection to the server and resend the request. This works unless chunking is on and the original data has already been flushed/streamed out to the socket and is not available for retransmission. With chunking we end up sending a empty body and not getting much of a reply so we timeout. (I Have not traced exactly what gets sent on the wire in this case but from the code it appears to just be the header, no body to which we get a continue)
The last bit of the puzzle is org.apache.cxf.transport.http.HTTPConduit.WrappedOutputStream.handleHeadersTrustCaching(). This code forces the early write of the headers which in the case of streaming/chunking means they will be lost. I think the onset of these problems must have coincided with the addition of handleHeadersTrustCaching?
The upshot of all this is that disabling chunking via config is a viable workaround. The problem is on the client, not on the server. Adding the following in cxf.xml and providing the config file name to createBus() does the trick.
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:http="http://cxf.apache.org/transports/http/configuration"
xsi:schemaLocation="
http://cxf.apache.org/transports/http/configuration http://cxf.apache.org/schemas/configuration/http-conf.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<http:conduit name="*.http-conduit">
<http:client AllowChunking="false"/>
</http:conduit>
</beans>
Also, disabling keepAlive at the HTTPClient layer has the desired effect: -Dhttp.keepAlive=false but we are back into system property land which should be avoided for specific test cases.
I guess the obvious question now surrounds handleHeadersTrustCaching() with chunking and keepalives and server closed sockets, do we expect this to work and if so can we fix it?
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Assignee: willem Jiang
> Attachments: busShutdownTest.patch.txt, jetty6.1.6.patch
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544486 ]
Gary Tully commented on CXF-1217:
---------------------------------
Re Jetty and SO_REUSEADDR, I just traced through the code and I think the problem may be that the SO_REUSEADDR option is set after the bind call, so it is only in effect for the next call on that channel/socket which will never ocurr in my test case. We get another Connector as we should.
There is no space between the open and bind call into which we can inject the SO_REUSEADDR property in org.mortbay.jetty.nio.SelectChannelConnector:
public void open() throws IOException
{
synchronized(this)
{
if (_acceptChannel == null)
{
// Create a new server socket
_acceptChannel = ServerSocketChannel.open();
// Bind the server socket to the local host and port
InetSocketAddress addr = getHost()==null?new InetSocketAddress(getPort()):new InetSocketAddress(getHost(),getPort());
_acceptChannel.socket().bind(addr,getAcceptQueueSize());
// Set to non blocking mode
_acceptChannel.configureBlocking(false);
}
}
}
Also, _acceptChannel is private so it is not easy to override this behaviour. Using a ServerSocketChanell provider is not really an option as it is difficult to tie this to the soReuseAddr option in the the CXF org.apache.cxf.transport.http_jetty.JettyHTTPServerEngine.
I am thinking that there should be a call to configureBind() or something, between the open and the bind calls.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Attachments: busShutdownTest.patch.txt
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
willem Jiang reassigned CXF-1217:
---------------------------------
Assignee: willem Jiang
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Assignee: willem Jiang
> Attachments: busShutdownTest.patch.txt
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Tully updated CXF-1217:
----------------------------
Attachment: jetty6.1.6.patch
a patch that requires jetty 6.1.6 (not yet in a repo, 6.1.6rc1 is) that ensures we set our version of ReuseAddr before it is need by jetty using the AbstractConnector. This will allow SO_REUSEADDR to be truned off as the default is on.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Assignee: willem Jiang
> Attachments: busShutdownTest.patch.txt, jetty6.1.6.patch
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544498 ]
Gary Tully commented on CXF-1217:
---------------------------------
This "open" call has been fixed in 2.1.6: http://svn.codehaus.org/jetty/jetty/tags/jetty-6.1.6/modules/jetty/src/main/java/org/mortbay/jetty/nio/SelectChannelConnector.java so we may be in luck with an upgrade once the option is set by CXF in the right place.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Attachments: busShutdownTest.patch.txt
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544143 ]
willem Jiang commented on CXF-1217:
-----------------------------------
After tracing the log , I found we did not really turn off the "org.apache.cxf.transports.http_jetty.DontClosePort" flag in the parent pom.xml.
When you run the test from Eclipse the flag is not set , the jetty engine will be shutdown when we call the bus.shutdown() in @After method.
Since we still set the "org.apache.cxf.transports.http_jetty.DontClosePort" true in the parent pom.xml, the jetty engine will not be shut down instead.
After I truned off the "org.apache.cxf.transports.http_jetty.DontClosePort" , I found there were dozens of systests failed. Now we still need to use "org.apache.cxf.transports.http_jetty.DontClosePort" flag in our parent pom.xml.
If we want to close the port when call the bus.shutdown(false), we still need set the "org.apache.cxf.transports.http_jetty.DontClosePort" to be false.
Willem.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544968 ]
Gary Tully commented on CXF-1217:
---------------------------------
Hi Willem, you may have committed too much!
I found the 'one' of AllowChunking="false" OR -Dhttp.keepAlive=false works for the BusShutdownTest. The problem is the interation of keepAlive and chunking, so turning off either should be sufficient. Apologies for not being more clear.
My preference would be for local versions of cxf.xml in the tests that need it rather than adding the configuration to all system tests.
---
On the questions: the keepAlive check interval is not set in HTTPConduit, it is set via sun.net.www.http.HttpClient and sun.net.www.http.KeepAliveCache.put(URL, Object, HttpClient). It is set via the response keep-alive header if present and defaults to 5seconds.
Having the server set a keep-alive header timeout value to say 1 second would also have the effect of ensuring that a closed connection is not used but this would impact on the value of keep-alive.
The reason a sleep(4000) works is because the first connection is cached in the keepAlive cache but gets cleaned out every 5 seconds. It is placed in the cache once the response is read. In the mean time, a bus is shutdown, sleep(...) and a new bus are created and 5 seconds pass. The keepAlive cache is cleared of any connection that has not been used in 5 seconds.
When the second invocation is made, there is nothing in the keepalive cache and a new connection is created as normal.
The problem arises when the second invocation uses a connection from the keepAliveCache that is partially closed, ie: the server has done a close. The client has to close it when the read fails and thus has to retry by resending the request on a new connection. When streaming/chunking is enabled however, the full request has not been cached and only the http headers of the request are sent, there is no body. The subsequent read hangs as the server is still waiting for the end/rest of the retransmitted request.
Does this make sense?
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Assignee: willem Jiang
> Attachments: busShutdownTest.patch.txt, jetty6.1.6.patch
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544949 ]
willem Jiang commented on CXF-1217:
-----------------------------------
Hi Gary,
Thank you for tracing this issue.
I just committed the code which fixed the systest errors when turn off the DontClosePort flag by setting the keep alive property and AllowChunking to be false.
Just two quick question , I did not find the code in the HTTPConduit which set the keepalive check interval. Can you point me out ?
Why can I get the test pass by waiting for a few second after the bus shutdown ? I do not think it is relate to the HTTPConduit retransmission mechanism.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Assignee: willem Jiang
> Attachments: busShutdownTest.patch.txt, jetty6.1.6.patch
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12545336 ]
willem Jiang commented on CXF-1217:
-----------------------------------
Hi Gray,
Thank you for the explanation, it make me much clear about the who HTTP keepAlive thing.
I tried to just turn off the AllowChunking , several MTOM and RM system tests failed, so I had to setup the keepalive system property for these test.
BTW, it is not easy to turn off the AllowChunking for the partial system tests without changing the code (let the bus to load the configure file), so I drop the cxf.xml into to the test resource directory.
Willem.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Assignee: willem Jiang
> Attachments: busShutdownTest.patch.txt, jetty6.1.6.patch
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "Benson Margulies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544458 ]
Benson Margulies commented on CXF-1217:
---------------------------------------
Gary,
I am a very, very, very, happy hijack-ee here.
--benson
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Attachments: busShutdownTest.patch.txt
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "Benson Margulies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544455 ]
Benson Margulies commented on CXF-1217:
---------------------------------------
When we last discussed this, there were two issues.
One was that CXF intentionally manages jetty engines with their own lifecycle. The goal is to permit multple CXF buses to publish endpoints on the same port(s) by sharing engines. So, a bus could shut down and leave an engine behind, \if something else is using the engine/.
My personal opinion is that this is not a good idea, but that's not a very important consideration. I think I left a JIRA hanging around asking for some explicit way to turn this off, in any case.
As I understand it, this design should not effect the cases in question here, since there shouldn't be something else holding the engine open.
I don't know the IBM JDK story.
There were bugs in the past in which the finger-counting went wrong, and the engine would 'stay alive' for no reason.
There was also rather a strong belief, based on some very strong evidence, that SO_REUSEADDR was \not/ the default state of Jetty's sockets. Willem put work into adding a code path that would explicitly set this option. The next version of Jetty has a much simpler API for that purpose, to boot.
Gary, do you know anything about the specifics of SO_REUSEADDR in Jetty?
One possibility is to write diagnostic code to read the state out.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Attachments: busShutdownTest.patch.txt
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544708 ]
willem Jiang commented on CXF-1217:
-----------------------------------
Hi Gary
I tried to upgrade the Jetty and used the setReuseAddress flag in AbstractConnector,
if I don't add the thread.sleep(4000) after the bus.shutdown, the client can't get the response message from server.
I do not want to add the sleep into the bus.shutdown, since it will consume lots of our systest time ,
I still suggest to use the org.apache.cxf.transports.http_jetty.DontClosePort property in our test.
BTW, I will commit you test case into the svn repository for further verification ;)
Willem.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Assignee: willem Jiang
> Attachments: busShutdownTest.patch.txt
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544446 ]
Gary Tully commented on CXF-1217:
---------------------------------
I don't fully understand the reason for org.apache.cxf.transports.http_jetty.DontClosePort but IMHO the idea of not shutting down the http transport is wrong. Is it possible to track down the root cause of the problem and deal with the bug there, where ever it is?
It should be possible to always set SO_REUSEADDR(default behaviour for server sockets) and have a server socket open/bind/close repeatidly in the same JVM. No?
Gary.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CXF-1217) Address in use errors from attempt to
repeatedly start and stop services
Posted by "willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CXF-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544685 ]
willem Jiang commented on CXF-1217:
-----------------------------------
Since I can't get Jetty 6.1.6 from maven , I will try to use Jetty 6.1.6 rc1.
Thanks Gray for tracing the code.
> Address in use errors from attempt to repeatedly start and stop services
> ------------------------------------------------------------------------
>
> Key: CXF-1217
> URL: https://issues.apache.org/jira/browse/CXF-1217
> Project: CXF
> Issue Type: Bug
> Components: Transports
> Affects Versions: 2.1
> Environment: Any
> Reporter: Benson Margulies
> Assignee: willem Jiang
> Attachments: busShutdownTest.patch.txt
>
>
> Edit org.apache.cxf.javascript.JsHttpRequestTest to take out the @org.junit.Ignore from the test case that has one, and run from eclipse. Watch the logging traffic. Observe:
> WARNING: failed Server@1174031
> java.net.BindException: Address already in use: bind
> at sun.nio.ch.Net.bind(Native Method)
> which eventually, as far as I can tell, turns into:
> INFO: doOpen POST http://localhost:8808/Greeter false
> Nov 18, 2007 10:07:08 PM org.apache.cxf.javascript.JsXMLHttpRequest communicate
> SEVERE: IO error reading response
> java.io.FileNotFoundException: http://localhost:8808/Greeter
> On the other hand, all is well in Eclipse.
> I will probably commit a workaround to this soon, so it would be good to move this muddle to its own test case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.