You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Laurent Petit <lp...@yseop.com> on 2012/07/02 10:59:53 UTC
Re: Re : Re: Issue with keep-alive connections, when using APR
Connector on Windows and starting Processes from Servlets
On Fri, 2012-06-29 at 17:18 +0200, Jeff MAURY wrote:
> Hello,
>
> I found the correct code to look at:
> http://svn.apache.org/repos/asf/tomcat/native/branches/1.1.x/native/src/network.c
> It does not set the inherit flag but it is using the apr library that maybe
> set the flag under Windows.
> If this is the case and according to your references, it is likely the
> client will not receive the ack and this is a problem with APR and
> Runtime.exec
Hello,
Seems like it's function apr_socket_create, defined in this APR file:
http://svn.apache.org/repos/asf/apr/apr/trunk/network_io/win32/sockets.c
which apparently already tries to create a non-inherited handle.
The code, for reasons beyond my current understanding, relies at compile
time on the flags APR_HAS_UNICODE_FS and _WIN32_WCE.
It also relies at runtime on IF_WIN_OS_IS_UNICODE, which may (?) be an
indirect indication that the underlying OS needs SetHandleInformation to
be explicitly called.
Jeff, do you have ideas/indications on how I could help push the issue
resolution forward ? We're reaching the limits of my current abilities
to help, not having a windows box with C compiler to do tests, etc. :-(
You'll find below the relevant code:
#ifdef WIN32
/* Socket handles are never truly inheritable, there are too many
* bugs associated. WSADuplicateSocket will copy them, but for our
* purposes, always transform the socket() created as a
non-inherited
* handle
*/
#if APR_HAS_UNICODE_FS && !defined(_WIN32_WCE)
IF_WIN_OS_IS_UNICODE {
/* A different approach. Many users report errors such as
* (32538)An operation was attempted on something that is not
* a socket. : Parent: WSADuplicateSocket failed...
*
* This appears that the duplicated handle is no longer
recognized
* as a socket handle. SetHandleInformation should overcome
that
* problem by not altering the handle identifier. But this
won't
* work on 9x - it's unsupported.
*/
SetHandleInformation((HANDLE) (*new)->socketdes,
HANDLE_FLAG_INHERIT, 0);
}
#if APR_HAS_ANSI_FS
/* only if APR_HAS_ANSI_FS && APR_HAS_UNICODE_FS */
ELSE_WIN_OS_IS_ANSI
#endif
#endif
#if APR_HAS_ANSI_FS || defined(_WIN32_WCE)
{
HANDLE hProcess = GetCurrentProcess();
HANDLE dup;
if (DuplicateHandle(hProcess, (HANDLE) (*new)->socketdes,
hProcess,
&dup, 0, FALSE, DUPLICATE_SAME_ACCESS)) {
closesocket((*new)->socketdes);
(*new)->socketdes = (SOCKET) dup;
}
}
#endif
#endif /* def WIN32 */
>
> Jeff
>
>
> On Fri, Jun 29, 2012 at 4:38 PM, Laurent Petit <lp...@yseop.com> wrote:
>
> > On Fri, 2012-06-29 at 15:27 +0200, Jeff MAURY wrote:
> > > No,
> > >
> > > it does not (yet) make sense because one piece of the puzzle is missing.
> > > The Local socket is not a TCP socket, it is created from a path rather
> > than
> > > from an IP address.
> > > So this socket is in no way linked to the browser, at least directly, and
> > > the missing piece is what is the relation from this socket to the
> > browser.
> >
> > You're right, so the question is still open.
> >
> > In the mean time, I'm come across interesting google search results,
> > among which the first one seems really interesting:
> >
> >
> > Oracle Bug 6428742
> > ==================
> >
> > "Currently, on Windows, Runtime.exec is implemented by calling
> > CreateProcess with the bInheritHandles argument set to TRUE,
> > which causes open handles to not be closed if they are not
> > explicitly set to be non-inherited. This is the underlying cause
> > of many subtle bugs, such as
> >
> > 6347873: (so) Ports opened with ServerSocketChannel blocks when using
> > Runtime.exec
> > http://monaco.sfbay/detail.jsf?cr=6347873"
> >
> >
> >
> > ==> Note the "if they are not explicitly set to be non-inherited" =>
> > this seems like a game changer, since our implicit assumption was that
> > we would just look for code which would have accidentally set the handle
> > to be inherited, not the other way around.
> >
> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6428742
> >
> >
> >
> > QT Project
> > ==========
> >
> > "When using spawned processes, sockets opened by the client remain in
> > CLOSE_WAIT state after closing the connection. Sockets on the remote
> > side would be in FIN_WAIT_2 state and waiting for ACK.
> > When the spawned (child) process is closed, then ACK is sent and sockets
> > are finally moved to CLOSE state on both sides.
> > This seems to be caused by the default handle inheritance on Windows by
> > child processes."
> >
> > https://bugreports.qt-project.org/browse/QTBUG-4465
> >
> >
> >
> > Zeromq2 project
> > ===============
> >
> > "On Windows platform, in order to prevent child processes to inherit
> > parent sockets, SetHandleInformation
> > (
> > http://msdn.microsoft.com/en-us/library/windows/desktop/ms724935(v=vs.85).aspx)
> > should be called on the socket handle.
> >
> > This patch comes out from discussion at:
> > https://github.com/zeromq/clrzmq/issues/60"
> >
> > https://github.com/zeromq/zeromq2-x/pull/51
> > and
> > https://zeromq.jira.com/browse/LIBZMQ-366
> >
> >
> > I found way more interesting results, but I'll spare space and won't
> > paste them all in this thread
> > ( google search criteria : "windows inheriting socket handles" )
> >
> >
> > >
> > > Jeff
> > >
> > >
> > > On Fri, Jun 29, 2012 at 2:40 PM, Laurent Petit <lp...@yseop.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > On Fri, 2012-06-29 at 11:45 +0200, Jeff MAURY wrote:
> > > >
> > > > > That what I guessed but I don't understand everything.
> > > > > The code you are referencing is related to NTPipes and not sockets.
> > > >
> > > >
> > > > I'm not a C expert, but my bet is that the file name is misleading.
> > > >
> > > > Indeed, the
> > > >
> > > > con->sa.bInheritHandle = TRUE;
> > > >
> > > > statement appears in the definition driven by the following macro
> > > > expansion:
> > > >
> > > > TCN_IMPLEMENT_CALL(jlong, Local, create)(TCN_STDARGS, jstring name,
> > > >
> > > > jlong pool)
> > > >
> > > > And when the macro call TCN_IMPLEMENT_CALL(jlong, Local, create) is
> > > > preprocessed, it gives:
> > > >
> > > >
> > > > JNIEXPORT RT JNICALL Java_org_apache_tomcat_jni_##Local##_##create
> > > >
> > > > , thus the bInheritHandle = TRUE statement is defined in the following
> > > > function signature:
> > > >
> > > > JNIEXPORT RT JNICALL
> > > > Java_org_apache_tomcat_jni_##Local##_##create(TCN_STDARGS, jstring
> > name,
> > > > jlong pool)
> > > >
> > > > which is a function called by JNDI, and as I guess, by class
> > > > org.apache.tomcat.ini.Local, method create() :
> > > >
> > > >
> > > >
> > http://svn.apache.org/repos/asf/tomcat/native/branches/1.1.x/java/org/apache/tomcat/jni/Local.java
> > > >
> > > > And the javadoc for class Local is: "Local socket"
> > > > And for methode create :
> > > > /**
> > > >
> > > > * Create a socket.
> > > > * @param path The address of the new socket.
> > > > * @param cont The parent pool to use
> > > > * @return The new socket that has been set up.
> > > > */
> > > >
> > > >
> > > > It seems like the native code uses a pool, and my bet is that the pool
> > > > is responsible for trying to reuse the socket.
> > > >
> > > > And something like this may happen: the socket handle, inherited by the
> > > > child process, puts the socket in such a state that the pool either
> > > > can't, either doesn't want to reuse it.
> > > >
> > > > And then what happens is that on the other side of the socket, the
> > > > client continues to write HTTP requests, and nobody is listening.
> > > >
> > > > And then when the Tomcat's child process is killed, the socket is
> > > > finally closed somehow, the client browser notices it and creates a new
> > > > connection to the server and retries the HTTP request.
> > > >
> > > >
> > > > Does that make sense ?
> > > >
> > > >
> > > >
> > > > > So I'm
> > > > > not familiar with Tomcat Native implementation, but do you know the
> > > > global
> > > > > architecture ?
> > > > > What I guessed is that the Tomcat native stuff created with the
> > inherit
> > > > > flag, but even if it does, I don't see why it should not work if the
> > > > socket
> > > > > is closed after a timeout.
> > > > >
> > > > > Regards
> > > > > Jeff
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Jun 29, 2012 at 11:22 AM, Laurent Petit <lp...@yseop.com>
> > > > wrote:
> > > > >
> > > > > > Hello Jeff, Konstantin & all,
> > > > > >
> > > > > > On Mon, 2012-06-25 at 20:52 +0200, verlag.preisser@t-online.dewrote:
> > > > > > > Hello Jeff & all,
> > > > > > >
> > > > > > > > Von: Jeff MAURY <je...@jeffmaury.com>
> > > > > > > > Datum: Mon, 25 Jun 2012 18:46:02 +0200
> > > > > > >
> > > > > > > > Konstantin,
> > > > > > > >
> > > > > > > > your explanations are very interesting but unclear to me: what
> > do
> > > > you
> > > > > > > > call the inactivity timer ? When it is started ? After the
> > request
> > > > has
> > > > > > > > been processed by the servlet ? In that case, I see no
> > difference
> > > > > > > > between a servlet that launch a process and another one.
> > > > > > > > It seems to me that a process that is launched does not
> > inhererits
> > > > > > > > handles from its parent process but it's possible that under
> > > > Windows,
> > > > > > > > it's an option so it would be interesting to watch.
> > > > > > > >
> > > > > > > > Jeff
> > > > > > > >
> > > > > > >
> > > > > > > Sorry, I'm just a normal Tomcat user, and I don't know how
> > exactly
> > > > the
> > > > > > APR connector and its Timeout works, so I am unable to answer that.
> > > > > > >
> > > > > > >
> > > > > > > Howewer, I did some further observations:
> > > > > > >
> > > > > > > -When I perform a request to the servlet that opens wordpad.exe,
> > the
> > > > TCP
> > > > > > connection from Tomcat does not close after the timeout - even
> > when I
> > > > kill
> > > > > > the Tomcat process (java.exe), the TCP connection is still open.
> > If I
> > > > kill
> > > > > > wordpad.exe, then finally the connection is closed/aborted.
> > > > > > > -When I have 1 TCP connection open to Tomcat and the servlet
> > starts
> > > > the
> > > > > > little C program, Task manager shows that it has 11 handles.
> > > > > > > However, when I have 5 TCP connections open to Tomcat, and do the
> > > > > > request on one of them, Task maanger shows that the C program has
> > 15
> > > > > > handles - so four more handles when there are four more
> > connections to
> > > > > > Tomcat. All of that 5 TCP connections don't close until I kill that
> > > > process.
> > > > > > >
> > > > > > > That seems to me to be an indication that socket handles could be
> > > > > > inherited by the child processes that are startet by ProcessBuilder
> > > > from
> > > > > > tomcat.
> > > > > > >
> > > > > > > A msdn article mentions how to create a new process using
> > > > > > CreateProcess(); it also mentions that socket handles can be
> > inherited:
> > > > > >
> > http://msdn.microsoft.com/en-us/library/windows/desktop/ms724466.aspx
> > > > > > >
> > > > > > > However, as I don't have much knowledge about programming with
> > > > WinAPIs,
> > > > > > I don't know why those handles are inherited (the MSDN article
> > mentions
> > > > > > that a handle must be specified as inheritable when created, to
> > allow a
> > > > > > child process to inherit it). Maybe someone with more WinAPI/Tomcat
> > > > Native
> > > > > > knowledge can help here.
> > > > > >
> > > > > >
> > > > > > I also had the vague intuition that this related to handles
> > > > inheritence.
> > > > > > Your recent tests & research tend to make this hypothesis even more
> > > > > > appealing.
> > > > > >
> > > > > > I know nothing about tomcat-native implementation, but I was able
> > to
> > > > see
> > > > > > that the bInheritHandle member is set to true here in tomcat
> > native's C
> > > > > > code:
> > > > > >
> > > > > > lpetit:~/tmp/tomcat-native $ grep "bInheritHandle" -R *
> > > > > > native/os/win32/ntpipe.c: con->sa.bInheritHandle = TRUE;
> > > > > >
> > > > > > (
> > > > > >
> > > > > >
> > > >
> > http://svn.apache.org/repos/asf/tomcat/native/branches/1.1.x/native/os/win32/ntpipe.c
> > > > )
> > > > > >
> > > > > >
> > > > > > Jeff,
> > > > > >
> > > > > > What else could we do to help investigate / fix this issue ?
> > > > > >
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > --
> > > > > > Laurent
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > Regards,
> > > > > > > Konstantin Preißer
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > > > > For additional commands, e-mail: users-help@tomcat.apache.org
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > Laurent Petit
> > > > > >
> > > > > > Agence +33 (0)4 78 47 07 49
> > > > > >
> > > > > > Email lpetit@yseop.com
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Yseop apporte une réponse intelligente et individualisée à chacun
> > de
> > > > vos
> > > > > > clients
> > > > > >
> > > > > >
> > > > > >
> > > > > > www.yseop.com
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > > > For additional commands, e-mail: users-help@tomcat.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Laurent Petit
> > > >
> > > > Agence +33 (0)4 78 47 07 49
> > > >
> > > > Email lpetit@yseop.com
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Yseop apporte une réponse intelligente et individualisée à chacun de
> > vos
> > > > clients
> > > >
> > > >
> > > >
> > > > www.yseop.com
> > > >
> > > >
> > > >
> > >
> > >
> >
> > --
> > Laurent Petit
> >
> > Agence +33 (0)4 78 47 07 49
> >
> > Email lpetit@yseop.com
> >
> >
> >
> >
> >
> >
> >
> > Yseop apporte une réponse intelligente et individualisée à chacun de vos
> > clients
> >
> >
> >
> > www.yseop.com
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
> >
>
>
--
Laurent Petit
Agence +33 (0)4 78 47 07 49
Email lpetit@yseop.com
Yseop apporte une réponse intelligente et individualisée à chacun de vos
clients
www.yseop.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org