You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by "Ben Craig (JIRA)" <ji...@apache.org> on 2012/09/10 16:24:07 UTC
[jira] [Created] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Ben Craig created THRIFT-1690:
---------------------------------
Summary: Sockets and Pipe Handles truncated on Win64
Key: THRIFT-1690
URL: https://issues.apache.org/jira/browse/THRIFT-1690
Project: Thrift
Issue Type: Bug
Components: C++ - Library
Affects Versions: 0.9
Environment: 64-bit Windows
Reporter: Ben Craig
On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Jens Geyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497471#comment-13497471 ]
Jens Geyer commented on THRIFT-1690:
------------------------------------
Regarding "moving the anonymous pipe transport to its own class": I used the C# code to write the Delphi Version, first version was a nearly 1:1 port. It worked fine so far, but somehow I was not happy with it.
After three iterations I finally ended up with some more classes, but much smaller ones, the code became much clearer after I decided to split them up.
I did that after it became clear to me that there were really two classes merged into one, causing a lot of the methods being in fact two methods rolled into one:
{code}
if( is_anonymous_case)
{
a lot of code
}
else
{
even more code
}
{code}
However, I do not have enough practice with *NIX platforms yet to be able to give any advice. I just wanted to mention my experiences with the C# code (and I still think about refactoring that one as well ... maybe, not today).
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470941#comment-13470941 ]
Hudson commented on THRIFT-1690:
--------------------------------
Integrated in Thrift #550 (See [https://builds.apache.org/job/Thrift/550/])
THRIFT-1690
Non-linux OSes use int's in bsd socket calls as well. (Revision 1394931)
Result = ABORTED
roger : http://svn.apache.org/viewvc/?view=rev&rev=1394931
Files :
* /thrift/trunk/lib/cpp/src/thrift/transport/TServerSocket.h
* /thrift/trunk/lib/cpp/src/thrift/transport/TSocket.h
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Ben Craig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470318#comment-13470318 ]
Ben Craig commented on THRIFT-1690:
-----------------------------------
One decent short term possibility is to change those two #ifdef linux's to #ifndef _WIN32.
A longer term solution is to stop providing unnatural aliases. Turning int -> SOCKET on non-Windows is one example. Turning errno into WSAGetLastError is similarly bad on Windows.
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Pavlin Radoslavov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471103#comment-13471103 ]
Pavlin Radoslavov commented on THRIFT-1690:
-------------------------------------------
A follow-up on my earlier comment:
The proper solution would be to check
for "SOCKET" inside configure.ac and conditionally typedef it for
all systems that don't have it.
Apparently, both header files TSocket.h and
TServerSocket.h are installed on the system
(by default in /usr/local/include/thrift/transport).
As such, the conditional typedef must use symbols
that are pre-defined for the client software that
might include those header files.
I.e., it must be a symbol defined in some of the other Thrift
header files installed on the system, or a
symbol defined by the system itself.
The "#ifndef _WIN32" solution is fine, because it
belongs to the latter category.
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Ben Craig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469653#comment-13469653 ]
Ben Craig commented on THRIFT-1690:
-----------------------------------
Roger,
I will be glad to better follow process in the future. I'm not sure what I messed up with my last submission though. What should I do better in the future? Single .patch for the entire fix? Remove the commentary in the .patch file?
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Ben Craig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ben Craig updated THRIFT-1690:
------------------------------
Attachment: libthrift_warning_purge.patch
libthrift_pipe_size.patch
libthrift_pipe_size resolves pipe specific issues.
libthrift_warning_purge resolves a lot of issues. Specifics are mentioned in the prologue of the .patch itself.
This is related to THRIFT-1695. libthrift_warning_purge is also linked on THRIFT-1695.
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Attachments: libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Peace C (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497213#comment-13497213 ]
Peace C commented on THRIFT-1690:
---------------------------------
One of the goals was to abstract away the platform-specific code (#ifdefs and such) into the transport, allowing the calling code to be uniform across platforms. As it stands now, both the Pipe transport code has #ifdefs, and code that calls it needs to employ #ifdefs. The calling code shouldn't care how the transport implements it, just that it's a local transport, be it named pipes or domain sockets. Otherwise apps that use Thrift would needlessly require redundant code "if (Windows) -> use Pipe code, if (Unix) -> use dskt code".
I agree the #ifdefs are ugly and I wish they could be avoided. I think you'd agree that if they have to be there, it's better to abstract it away into the transport instead of placing that burden on every code that uses it.
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469586#comment-13469586 ]
Hudson commented on THRIFT-1690:
--------------------------------
Integrated in Thrift #543 (See [https://builds.apache.org/job/Thrift/543/])
THRIFT-1690 Sockets and Pipe Handles truncated on Win64
Patch: Ben Craig (Revision 1394182)
Result = FAILURE
roger : http://svn.apache.org/viewvc/?view=rev&rev=1394182
Files :
* /thrift/trunk/lib/cpp/src/thrift/concurrency/BoostMonitor.cpp
* /thrift/trunk/lib/cpp/src/thrift/concurrency/TimerManager.cpp
* /thrift/trunk/lib/cpp/src/thrift/concurrency/Util.cpp
* /thrift/trunk/lib/cpp/src/thrift/concurrency/Util.h
* /thrift/trunk/lib/cpp/src/thrift/protocol/TBinaryProtocol.tcc
* /thrift/trunk/lib/cpp/src/thrift/protocol/TDebugProtocol.cpp
* /thrift/trunk/lib/cpp/src/thrift/protocol/TDenseProtocol.cpp
* /thrift/trunk/lib/cpp/src/thrift/protocol/TJSONProtocol.cpp
* /thrift/trunk/lib/cpp/src/thrift/protocol/TProtocol.h
* /thrift/trunk/lib/cpp/src/thrift/server/TThreadPoolServer.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/TBufferTransports.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/TBufferTransports.h
* /thrift/trunk/lib/cpp/src/thrift/transport/TFDTransport.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/TFileTransport.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/TFileTransport.h
* /thrift/trunk/lib/cpp/src/thrift/transport/THttpClient.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/THttpServer.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/THttpTransport.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/TPipe.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/TPipe.h
* /thrift/trunk/lib/cpp/src/thrift/transport/TPipeServer.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/TPipeServer.h
* /thrift/trunk/lib/cpp/src/thrift/transport/TServerSocket.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/TServerSocket.h
* /thrift/trunk/lib/cpp/src/thrift/transport/TSocket.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/TSocket.h
* /thrift/trunk/lib/cpp/src/thrift/transport/TSocketPool.cpp
* /thrift/trunk/lib/cpp/src/thrift/transport/TSocketPool.h
* /thrift/trunk/lib/cpp/src/thrift/windows/SocketPair.cpp
* /thrift/trunk/lib/cpp/src/thrift/windows/SocketPair.h
* /thrift/trunk/lib/cpp/src/thrift/windows/StdAfx.h
* /thrift/trunk/lib/cpp/src/thrift/windows/WinFcntl.cpp
* /thrift/trunk/lib/cpp/src/thrift/windows/WinFcntl.h
* /thrift/trunk/lib/cpp/src/thrift/windows/config.h
* /thrift/trunk/lib/cpp/src/thrift/windows/force_inc.h
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Ben Craig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470408#comment-13470408 ]
Ben Craig commented on THRIFT-1690:
-----------------------------------
Hopefully that patch will do the trick. As mentioned on the comments, I haven't actually tested it on non-Linux, non-Windows builds.
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Ben Craig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497482#comment-13497482 ]
Ben Craig commented on THRIFT-1690:
-----------------------------------
So what is the expectation of source compatibility with Thrift? If I change TSocket and TServerSocket to be TCP only, TPipe and TServerPipe to be localhost named transports, and create TAnonymousPipe classes, would the code breaks be met with insurmountable resistance?
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Peace C (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496471#comment-13496471 ]
Peace C commented on THRIFT-1690:
---------------------------------
The fixes are very much appreciated. We're not building 64-bit yet but when the time comes this will be beneficial.
A consequence of the restructuring (wrapping the Pipe classes with #ifdef _WIN32 and typedef'ing the name for *NIX) is that it's more difficult to implement anonymous pipes for *NIX. It'll be alot of duplicated code since all the Pipe related methods will need to be replicated for *NIX. It also makes the calling code less platform-agnostic. With these changes, the calling code can't be cross-platform for Windows & *NIX because, for example, setPipename doesn't exist in TServerSocket. You've just moved the burden of the #ifdefs from the abstract Pipe class to the calling code. That was the motivation for casting HANDLE to int and vice-versa. Perhaps we could cast between IntPtr instead?
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Pavlin Radoslavov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pavlin Radoslavov reopened THRIFT-1690:
---------------------------------------
The patch (Revision 1394182) broke the compilation for non-Linux UNIX
systems, because SOCKET is not defined. The compilation error below
is on FreeBSD-9.0:
/bin/sh ../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../.. -I/usr/local/include -I./src -I./src/thrift -Wall -g -O2 -MT THttpClient.lo -MD -MP -MF .deps/THttpClient.Tpo -c -o THttpClient.lo `test -f 'src/thrift/transport/THttpClient.cpp' || echo './'`src/thrift/transport/THttpClient.cpp
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../.. -I/usr/local/include -I./src -I./src/thrift -Wall -g -O2 -MT THttpClient.lo -MD -MP -MF .deps/THttpClient.Tpo -c src/thrift/transport/THttpClient.cpp -fPIC -DPIC -o .libs/THttpClient.o
In file included from ./src/thrift/transport/TSocket.h:27,
from src/thrift/transport/THttpClient.cpp:25:
./src/thrift/transport/TServerSocket.h:64: error: 'SOCKET' has not been declared
./src/thrift/transport/TServerSocket.h:69: error: 'SOCKET' does not name a type
./src/thrift/transport/TServerSocket.h:79: error: 'SOCKET' does not name a type
./src/thrift/transport/TServerSocket.h:80: error: 'SOCKET' does not name a type
In file included from src/thrift/transport/THttpClient.cpp:25:
./src/thrift/transport/TSocket.h:203: error: 'SOCKET' does not name a type
./src/thrift/transport/TSocket.h:234: error: expected `)' before 'socket'
./src/thrift/transport/TSocket.h:265: error: 'SOCKET' does not name a type
gmake[4]: *** [THttpClient.lo] Error 1
gmake[4]: Leaving directory `/usr/home/pavlin/tmp/thrift/thrift-svn/lib/cpp'
gmake[3]: *** [all-recursive] Error 1
gmake[3]: Leaving directory `/usr/home/pavlin/tmp/thrift/thrift-svn/lib/cpp'
gmake[2]: *** [all-recursive] Error 1
gmake[2]: Leaving directory `/usr/home/pavlin/tmp/thrift/thrift-svn/lib'
gmake[1]: *** [all-recursive] Error 1
gmake[1]: Leaving directory `/usr/home/pavlin/tmp/thrift/thrift-svn'
gmake: *** [all] Error 2
Exit 2
The root cause is the following conditional definition inside file TSocket.h:
#ifdef __linux__
typedef int SOCKET;
#endif
Obviously, this is not portable. The proper solution would be to check
for "SOCKET" inside configure.ac and conditionally typedef it for
all systems that don't have it.
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Ben Craig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496527#comment-13496527 ]
Ben Craig commented on THRIFT-1690:
-----------------------------------
The old code was a bit of a portability mess. The new code is a different portability mess. The toy program that I've been using for my own testing has gotten away without needing to touch the raw HANDLE / file descriptor. I would prefer to solve most use cases in that way, but still provide access to the underlying OS object when platform specific things are needed. I think #ifdefs are inevitable there. There is still significant room to improve though. I don't think IntPtr is the answer though, as that is too large on 64-bit *nix platforms.
The projects I am working on care a lot about named pipes and named AF_UNIX sockets. This is for Mac, Windows, and Linux, both 32 and 64-bit. I am actively working on making it easy to use these in cross-platform code. I hope to have patches for the following problems in the next month:
1. Portable naming. I want to be able to construct my TTransport (or TServerTransport) without ifdefs. That means that "MyPipe" needs to have the appropriate slashes added on Windows (already done). It also means that "MyPipe" needs to either be put in a special folder on *nix, or needs to be an abstract socket. Putting the socket file in the app's current working directory probably isn't desirable.
2. Better cleanup. Stop doesn't currently work on TPipeServer. Stopping AF_UNIX sockets doesn't clean up the file (and I'm not sure it can safely). Both of these cases are important to me, because my use cases aren't of the "launch the app and let it run 2 years" variety. My clients and servers start and stop in interactive scenarios.
I don't have a personal need / interest in anonymous pipes. I wasn't trying to break them, but I also didn't have tests for them. I think the correct way to handle these is to have a new TServerTransport class specifically for anonymous pipes. It may forward to TPipeServer and TServerSocket, but it can have the constructors that make sense for anonymous pipes, and not carry the baggage of named pipes.
For the record, I'm not thrilled with the "multi-purpose" transports like TSocket, TPipe, and their server counterparts. Porting single aspects of those classes is normally easy, but porting all of the aspects often doesn't work. My preferred approach would have involved TTCPSocket, TNamedSocket(?), TAnonymousSocket, and the like. A lot of them would have similar implementations (possibly forwarding to helper classes), but it would make for more portable client code.
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Roger Meier (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roger Meier resolved THRIFT-1690.
---------------------------------
Resolution: Fixed
Thanks Craig for this quick fix!
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Peace C (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497239#comment-13497239 ]
Peace C commented on THRIFT-1690:
---------------------------------
PS: You're probably aware but just in case - that sample won't work with the current pipe transport. It'll require additional #ifdefs to set up the pipe transport on Windows (calls which don't exist in the *NIX pipe typedef).
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Roger Meier (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roger Meier reassigned THRIFT-1690:
-----------------------------------
Assignee: Roger Meier
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Roger Meier (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469709#comment-13469709 ]
Roger Meier commented on THRIFT-1690:
-------------------------------------
one patch per issue made acording to http://thrift.apache.org/docs/HowToContribute
would be great, diff's made within a subtree are not that easy to handle.
simplification, that's all...
thanks for the patch!
roger
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Ben Craig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ben Craig updated THRIFT-1690:
------------------------------
Attachment: lib_socket_typedef.patch
Fix for non-linux, non-windows builds of C++ library. Only tested on Linux and Windows, to make sure I didn't make things worse.
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Pavlin Radoslavov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470244#comment-13470244 ]
Pavlin Radoslavov commented on THRIFT-1690:
-------------------------------------------
Clarification to the above comment:
The same conditional typedef of SOCKET in TSocket.h
is repeated in file TServerSocket.h as well:
#ifdef _linux_
typedef int SOCKET;
#endif
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Ben Craig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497222#comment-13497222 ]
Ben Craig commented on THRIFT-1690:
-----------------------------------
I think we are agreeing (at least on some parts). I don't want client code to require #ifdefs. I think we might be disagreeing (or at least not understanding each other) on how to go about that.
My proposal (that I don't have immediate plans on implementing) is to have a TAnonymousPipe / TAnonymousSocket transport. I think that would provide the appropriate level of abstraction in client code.
I would like TPipe to be focused on named pipes / AF_UNIX sockets.
If you have a specific suggestion for a way to merge the two in a way that is not client hostile, I would be glad to review it. I am open to being proven wrong. Of course, in the end, I'm not the one you need to convince, as I'm not a maintainer :)
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Roger Meier (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470296#comment-13470296 ]
Roger Meier commented on THRIFT-1690:
-------------------------------------
sorry about that!
do you already have a patch?
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Roger Meier (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roger Meier resolved THRIFT-1690.
---------------------------------
Resolution: Fixed
Thanks Ben, committed!
Could you please create future patches according to
http://thrift.apache.org/docs/HowToContribute/
=> this simplifies and speeds up review/commit phase.
thanks
roger
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Peace C (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497495#comment-13497495 ]
Peace C commented on THRIFT-1690:
---------------------------------
>From what I can tell, alot of people rely on domain sockets in T[Server]Socket. Consider also that domain sockets are handled very similarly to TCP sockets so you'd end up with alot of redundant code from the socket transport.
I recommend reverting to the #ifdefs that were originally in the pipe transports to retain better cross-platform support. Pull out the anonymous pipe portion and place that into its own class. We'll need to solve how to cleanly cast HANDLE to/from 32 & 64-bit types since *NIX doesn't have HANDLEs. Could we typedef a 32/64-bit long to HANDLE on *NIX systems, assuming bitness can be detected by the preprocessor?
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1690) Sockets and Pipe Handles truncated
on Win64
Posted by "Peace C (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497238#comment-13497238 ]
Peace C commented on THRIFT-1690:
---------------------------------
I don't have any objection to moving the anonymous pipe transport to its own class. There wouldn't be an anonymous socket though (domain sockets are always named). On *NIX, an unnamed pipe can be created and the resulting file R/W descriptors passed on to the child process, conceptually similar to Windows' handles created from CreatePipe.
Take a look at contrib/transport-sample for cross-platform code that uses pipe transport. There are still some #ifdefs in there, many of which can't be avoided simply due to general platform differences. A portion of the #ifdefs are in the anonymous pipe section which could potentially be abstracted.
Upon reviewing it, I see a few potential things that could be pulled into the transport. For instance, there's an unlink called in RunThriftServer in ThriftCommon.cpp. It's there in case the same domain socket had been previously used. Perhaps that could be placed in the destructor to address the cleanup concerns?
> Sockets and Pipe Handles truncated on Win64
> -------------------------------------------
>
> Key: THRIFT-1690
> URL: https://issues.apache.org/jira/browse/THRIFT-1690
> Project: Thrift
> Issue Type: Bug
> Components: C++ - Library
> Affects Versions: 0.9
> Environment: 64-bit Windows
> Reporter: Ben Craig
> Assignee: Roger Meier
> Attachments: lib_socket_typedef.patch, libthrift_pipe_size.patch, libthrift_warning_purge.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> On 64-bit Windows, "int" is a 32-bit value. SOCKET and HANDLE are 64-bit.
> All of the files dealing with sockets in thrift use "int" as the type of a socket, as this is the idiomatic way to handle sockets on POSIX systems. For portability, a SOCKET typedef is probably needed.
> For the Pipe Server and Pipe Transport, HANDLEs are cast to ints to store as member variables for some reason (maybe to avoid #including <windows.h> in a header?).
> Both of these situations can result in invalid handles being used (and valid handles being leaked) when the system is under load.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira